microsoft.com2023-12-12officialauthorityHN front pageHN 269 pts · 121 comments1,064 words
Microsoft Research released Phi-2, a 2.7 billion-parameter language model available in the Azure AI Studio model catalog. The model was trained on 1.4T tokens from multiple passes on a mixture of Synthetic and Web datasets for NLP and coding. Training for Phi-2 took 14 days on 96 A100 GPUs, and the base model has not undergone alignment through reinforcement learning from human feedback (RLHF) nor has it been instruct fine-tuned.written, reviewed, cited
- Phi-2 matches or outperforms models up to 25x larger on complex benchmarks due to new innovations in scaling and data curation.
- The model achieves better behavior regarding toxicity and bias compared to existing open-source models that underwent alignment through RLHF.
- Phi-2 surpasses the performance of Mistral and Llama-2 models at 7B and 13B parameters on various aggregated benchmarks.
techcommunity.microsoft.com2023-11-15officialauthorityHN 7 pts · 6 comments11,693 words
Microsoft adds Mistral, Phi, Jais, Code Llama, and NVIDIA Nemotron to the Azure AI Model Catalog and Models as a Service. Jais is a 13-billion parameter model trained on the Condor Galaxy 1 AI supercomputer, while Mistral uses grouped query attention with 7.3 billion parameters. MaaS enables fine-tuning without managing GPU infrastructure, and the catalog now includes 40 new models.written, reviewed, cited
- Jais inference APIs and hosted fine-tuning are available without requiring users to manage GPU infrastructure.
