Keep Your IP Private: Using Local AI Models in VS Code
by Stephen Hewitt • 5 min read
Sometimes you can't use off-site AI like ChatGPT or Claude due to concerns about having a third party accessing your customer's Intellectual Property. At Sembit, we don't use such tools unless our customer has explicitly approved the use of AI in creating their software.
However, there's another option - you can run an AI model locally. That way you know there is no private IP being sent over the internet. Here's one way to do it:
- In Visual Studio Code, install the Continue extension. This is an AI plugin that can work with local LLMs.
- On your computer, install Ollama, which is tooling to run LLMs locally.
- When Ollama is running, use the command line to download an appropriately-sized model. Smaller models are less accurate, but run faster and use less GPU memory. Here is how you can download and run a small model specifically trained for coding:
ollama run qwen2.5-coder:1.5b
- Configure the Continue extension to use the local ollama model in your project by creating a .continue/config.json file in the project's root folder. Here's an example of the config file contents. The first section is for chat, and the second section is for auto-complete:
{
"models": [
{
"title": "Qwen2.5-Coder 1.5B",
"provider": "ollama",
"model": "qwen2.5-coder:1.5b-base"
}
],
"tabAutocompleteModel": {
"title": "Qwen2.5-Coder 1.5B",
"provider": "ollama",
"model": "qwen2.5-coder:1.5b-base"
}
}
Here is an example of what you might expect from this small coding model, and what that looks like in VS Code:
>Prompt
using vanilla javascript, how can i find a table by id, get all of
its rows, and loop through the rows?
Response from qwen2.5-coder:1.5b
In JavaScript using vanilla, you can find an element by ID, retrieve
all its rows from a `<table>` element, and then iterate over them
using a `forEach`
loop. Here's a step-by-step example:
```javascript
// Step 1: Find the table element by its ID
var tableElement = document.getElementById('myTable');
// Step 2: Get all of the rows in the table
var rows = tableElement.rows;
// Step 3: Loop through each row
rows.forEach(function(row, index) {
// Access the cells in the current row
var cells = row.cells;
// Example: Log the text content of the first cell in each row
console.log("Row " + (index + 1) + ": " + cells[0].textContent);
});
```
In this example:
- `document.getElementById('myTable')` retrieves the `<table>` element
from the HTML document with the ID `'myTable'`.
- `.rows` gives you a collection of all rows in the table.
- The `forEach` function is used to iterate over each row, and within
the loop, you can access its cells and perform operations on them.
This method allows you to easily traverse through the data in a table
using JavaScript.
Comments
The above results took less than 3 seconds to generate. Smaller models run quickly. However, if you look closely, you'll see that it isn't exactly right - there is not a "forEach" function on the rows property of the table element. That was hallucinated.
Response from qwen2.5-coder:32b (same prompt)
Certainly! To achieve this using vanilla JavaScript, you can follow
these steps:
1. **Find the Table**: Use `document.getElementById()` to select the
table by its ID.
2. **Get All Rows**: Access the `rows` property of the table element,
which returns an HTMLCollection of all `<tr>` elements within the
table.
3. **Loop Through Rows**: Use a loop (e.g., `for` or `forEach`) to
iterate through each row.
Here's a sample code snippet that demonstrates these steps:
```html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-
scale=1.0">
<title>Table Row Example</title>
</head>
<body>
<table id="myTable">
<tr><td>Row 1, Cell 1</td><td>Row 1, Cell 2</td></tr>
<tr><td>Row 2, Cell 1</td><td>Row 2, Cell 2</td></tr>
<tr><td>Row 3, Cell 1</td><td>Row 3, Cell 2</td></tr>
</table>
<script>
// Step 1: Find the table by its ID
var table = document.getElementById('myTable');
if (table) {
// Step 2: Get all rows from the table
var rows = table.rows;
// Step 3: Loop through each row
for (var i = 0; i < rows.length; i++) {
var row = rows[i];
// Do something with each row, e.g., log its innerHTML
to the console
console.log('Row ' + (i + 1) + ': ', row.innerHTML);
}
} else {
console.log('Table not found');
}
</script>
</body>
</html>
```
### Explanation:
- **`document.getElementById('myTable')`**: Retrieves the table
element with the ID `myTable`.
- **`table.rows`**: Returns a collection of all `<tr>` elements in the
table.
- **`for (var i = 0; i < rows.length; i++)`**: Iterates over each row
in the collection.
### Alternative Using `forEach`:
If you prefer using `forEach`, note that HTMLCollections do not have a
`forEach` method directly. However, you can convert it to an array
first:
```javascript
// Convert the HTMLCollection to an array
var rowsArray = Array.from(table.rows);
// Use forEach to loop through each row
rowsArray.forEach(function(row, index) {
console.log('Row ' + (index + 1) + ': ', row.innerHTML);
});
```
This approach is more modern and can be more readable for some
developers.
Comments
The results for the same prompt from the much larger 32B parameter model took about a minute to generate. It gave two options to loop through the rows of the table, both with correct syntax. If you have enough memory and are willing to wait, you will get more accurate results from a larger model. In this particular case, you might prefer a faster response if you can trivially fix the forEach goof that the smaller model made.
Conclusion
The most powerful models run on large machines in data centers, but when that's not an option, you can still make use of AI assistance by running smaller models on your own computer. Try various options, and see what works for your setup. If you want to discuss using AI for your project, contact Sembit - we're happy to help!
