In my view, AI is going to be more valuable in helping programmers understanding existing code rather than generating new code. Here @judell explores using AI to help write documentation
https://thenewstack.io/code-in-context-how-ai-can-help-improve-our-documentation/
@mfowler I literally spent last night feeding the C# class I wrote into a local AI and it documented it all with XMLDoc style comments. Got all that Visual Studio hover-for-information-goodness for free. And this was just a tiny 7B Mistral model. I'm blown away, truly.
Could I have written that myself? Sure, but that requires extra brain power for prose and explanations that I'd rather put toward the code.
@wagesj45 @mfowler Are you able to share any more on how this is done?
Personally, I think it's vital that this is a local process, but I've found the information out there about getting local models to ingest reference material is non-existent.
I'm just about to take ownership of python code based that is a few 10s of thousands of lines without documentation. Could be a good application of the tech.
@weebull The only "complex" thing is that I have a Linux server in my basement that I use to run oobabooga text-generation-webui. The model I'm running is Mistral-7B-Instruct-v0.2 (the 5-bit quantization from TheBloke on huggingface).
Then I basically just paste my code into the chat function. Something like this...
I want you to look at this C# class and analyze it. Then I want you to write the XMLDoc comments for the code. [extra instructions here].
```csharp
[code]
```
@weebull for what I'm doing so far, it has been adequate. The more context and the more parameters in the model, the better. Mistral is overpowered for its size/speed, but it's also not something I'd put into production. If you are serious about using AI in this local context and also need it for enterprise, you're going to need a LOT of horsepower to run a bigger model and maybe even do some fine tuning.
I don't think you'll find a "drop in" solution, but it'll be an adventure!