Behind various popular app development tools, Jetbrains has released its first “open” AI model for coding.
On Wednesday, JetBrains created Mellum, a code generation model released last year for various software development suites. Trained with over 4 trillion tokens, mellum weighs 4 billion parameters and is specifically designed for code completion (i.e., complete code snippets based on surrounding context).
The parameters roughly correspond to the model’s problem-solving skills, but the tokens are raw bits of the data the model processes. A million tokens correspond to approximately 30,000 lines of code.
“Designed for integration into professional developer tools (intelligent code proposals in an integrated developer environment), AI-powered coding assistants, and research into code understanding and generation, Mellum is also suitable for educational applications and fine-tuning experiments.”
Jetbrains says it trained Mellum with Apache 2.0 licenses on a collection of datasets containing acceptable license codes from Github and Wikipedia articles in English. It took about 20 days to train on a cluster of 256 H200 NVIDIA GPUs.
Mellum does some work to get up and run. The base model must not be out of the box. You need to tweak it first. Jetbrians offers several fine-tuned mellum models to Python, but the company warns that it intends to “estimate potential capabilities” that won’t be deployed in production environments.
The code generated by AI will undoubtedly change the way the software is built, but it also introduces new security challenges. A survey from developer security platform Synk in late 2023 shows that over 50% of organizations may encounter security issues with AI-enacted code.
TechCrunch Events
Berkeley, California
|
June 5th
Book now
In fact, Jetbrains points out that mellum “reflects the biases that exist in the public codebase” (e.g., generates code similar to open source repositories), and that suggestions for that code are not necessarily “safe or vulnerable.”
“This is just the beginning,” Jetbrains wrote in a blog post. “We’re not chasing generality. We’re focused. If Mellum causes meaningful experimentation, contribution, or even collaboration, we consider it a victory.”