I have a personal use copy of Mathematica, and wanted to play with the new LLM functionality (LLMConfiguration, LLMFunction, …)
It turns out that this functionality is doubly useless. Wolfram wants to charge you extra to use it with their models, and it fails with the chatGPT models, because openAI also requires a subscription payment tier.
However, I found that you can install and run local LLM models fairly easily. Using ollama, it was fairly trivial to install and run the llama2 and mixtral models. I was surprised just how easy it is to get an LLM running on your own hardware.
The steps, if I remember them properly, was just:
- Install ollama
- ollama pull llama2
- ollama run llama2
Once you have the model’s installed, you can check that the model(s) are installed, and how much storage they consume:
# ollama list NAME ID SIZE MODIFIED mixtral:latest a3b6bef0f836 26 GB 31 hours ago llama2:latest 78e26419b446 3.8 GB 32 hours ago
Here’s an example of a command line session with the mixtral model:
# ollama run mixtral
>>> Show me Mathematica code to solve the linear system: a x + b y = c, d x + e y = f
Here is some Mathematica code that solves the linear system you have given:
```
a = 2; b = 3; c = 4;
d = 5; e = 6; f = 7;
sol = Solve[{a x + b y == c, d x + e y == f}, {x, y}]
(* {{x -> -(1/2) - (3 f)/14, y -> 2/7 + (5 c)/14}} *)
```
Incidentally, the llama2 model gets this wrong, and suggests:
which is malformed. The mixtral model needlessly includes specific values for all the matrix elements, but can be coached not to do so.
One nice thing about ollama is that it provides a rest server interface, so you can query it programatically. Here’s some sample python code cobbled together to do just that:
#! /usr/bin/env python3
import requests
import json
import sys
import os
import argparse
import re
from pathlib import Path
#MODEL = 'mixtral'
MODEL = 'llama2'
CACHE_FILE = ''
def load_cache():
if os.path.exists(CACHE_FILE):
with open(CACHE_FILE, 'r') as f:
return json.load(f)
return []
def save_cache(context):
with open(CACHE_FILE, 'w') as f:
json.dump(context, f)
def query_ollama(prompt, context):
response = requests.post(
'http://localhost:11434/api/generate',
json={'model': MODEL, 'prompt': prompt, 'context': context, 'stream': False}
).json()
return response
def main():
global MODEL, CACHE_FILE
# Set up argument parser
parser = argparse.ArgumentParser(description="Query the Ollama Mixtral LLM model.")
parser.add_argument('--clean', action='store_true', help="Start with an empty context list (no cache).")
parser.add_argument('--model', type=str, default=MODEL, help="Specify an alternative model.")
parser.add_argument('--cache', type=str, default='context_cache', help="context cache path component.")
parser.add_argument('--query', type=str, default='', help="multiline query in a file")
args = parser.parse_args()
MODEL = args.model
print(f"Using model: {MODEL}")
cache = args.cache;
CACHE_FILE = os.path.expanduser(f'~/.llmq.{cache}.{MODEL}.json')
print(f"Context cache file: {CACHE_FILE}")
# Initialize context based on command-line options
if args.clean:
context = []
else:
context = load_cache()
while True:
if args.query:
prompt = Path(args.query).read_text().strip()
else:
# Read input from stdin (interactive prompt)
prompt = input("Enter your query: ")
# Check if the prompt matches exit or quit
if re.match(r'^(exit|quit)$', prompt.lower()):
print("Exiting...")
break
# Query the model and get the response
response = query_ollama(prompt, context)
rr = response.get('response', 'No response key in API output')
print(f"Response: {rr}")
# Update context (for context propagation)
context = response.get('context', [])
save_cache(context)
if args.query:
break
if __name__ == "__main__":
main()
# vim: et ts=4 sw=4
This code initially queries the rest server with an empty list context, but has a mechanism to save context once returned. Here is a sample session with the script:
# llmq.py --model mixtral --cache mathematica
Using model: mixtral
Context cache file: /Users/pjoot/.llmq.mathematica.mixtral.json
Enter your query: show me code to plot sin(x), and 2 sin(x) cos(x) in a single plot with different colors
Response: To plot both `Sin[x]` and `2 Sin[x] Cos[x]` in the same plot with different colors, you can use the `Plot` function in Mathematica as follows:
```mathematica
Plot[{Sin[x], 2*Sin[x]*Cos[x]}, {x, -Pi, Pi}, PlotStyle -> {Blue, Red}]
```
This will create a plot with `Sin[x]` in blue and `2 Sin[x] Cos[x]` in red. The x-axis ranges from `-Pi` to `Pi`. The `Plot` function automatically chooses appropriate tick marks and axis labels, but you can customize these using additional options if desired.
Note that the option `PlotStyle -> {Blue, Red}` specifies that the first curve should be plotted in blue and the second curve in red. You can change the colors by modifying this option.[br]
You can also add labels and title to your plot with the following code:
```mathematica
Plot[{Sin[x], 2*Sin[x]*Cos[x]}, {x, -Pi, Pi}, PlotStyle -> {Blue, Red}, PlotLabel -> "Plots of sin(x) and 2 sin(x) cos(x)", AxesLabel -> {"x", ""}]
```
This will create a plot with the title "Plots of sin(x) and 2 sin(x) cos(x)" and labels "x" on the x-axis.[br]
You can also add a legend to your plot by adding the option `PlotLegends -> {"sin(x)", "2 sin(x)cos(x)"}`:
```mathematica
Plot[{Sin[x], 2*Sin[x]*Cos[x]}, {x, -Pi, Pi}, PlotStyle -> {Blue, Red}, PlotLabel -> "Plots of sin(x) and 2 sin(x) cos(x)", AxesLabel -> {"x", ""}, PlotLegends -> {"sin(x)", "2 sin(x)cos(x)"}]
```
Enter your query: Now I'd like to plot the gamma function in the complex plane. What is a good way to visualize that?
Response: Visualizing the Gamma function on the complex plane can be done using a 3D surface plot, where the real and imaginary parts of the input argument are the x-axis and y-axis, respectively, and the output value (the magnitude of the Gamma function) is the z-axis. Here's an example code in Mathematica to create such a plot:
```mathematica
ComplexPlot3D[Gamma[x + I*y], {x, -5, 5}, {y, -5, 5}]
```
This creates a 3D surface plot of the Gamma function on the complex plane for the range `-5 <= Re(z) <= 5` and `-5 <= Im(z) <= 5`. The resulting plot shows the magnitude of the Gamma function as a surface in 3D space, with contour lines indicating the values of the function.
Alternatively, you can use the following code to create a density plot of the absolute value of the Gamma function on the complex plane:
```mathematica
ComplexPlot[Abs[Gamma[x + I*y]], {x, -5, 5}, {y, -5, 5}]
```
This creates a 2D density plot of the absolute value of the Gamma function on the complex plane for the same range as before. The resulting plot shows the magnitude of the Gamma function using a color scale, with darker colors indicating higher values and lighter colors indicating lower values. This can be useful to visualize the behavior of the Gamma function in different regions of the complex plane.
Note that the Gamma function has poles at non-positive integers, so the plot will show singularities at those points. You may want to exclude these points from your plot by using `Exclusions -> {Re[z] <= 0, Im[z] == 0, Re[z] == Integer}` in the `ComplexPlot3D` function or `Exclusions -> {Re[z] <= 0, Im[z] == 0}` in the `ComplexPlot` function.
Enter your query: that produced the error: Options expected (instead of {y,-5,5}) beyond position 2 in ...
Response: I apologize for the mistake in my previous response. Here's the corrected code to create a density plot of the absolute value of the Gamma function on the complex plane:
```mathematica
ComplexPlot[Abs[Gamma[x + I*y]], {x, -5, 5}, {y, -5, 5}]
```
This creates a 2D density plot of the absolute value of the Gamma function on the complex plane for the range `-5 <= Re(z) <= 5` and `-5 <= Im(z) <= 5`. The resulting plot shows the magnitude of the Gamma function using a color scale, with darker colors indicating higher values and lighter colors indicating lower values. This can be useful to visualize the behavior of the Gamma function in different regions of the complex plane.
Again, I apologize for any confusion my previous response may have caused.
Enter your query: from the manual it appears that the plot range should be complex: like this : {z, -2 - 2 I, 2 + 2 I}
Response: You're absolutely right! Thank you for bringing that up. When creating a `ComplexPlot` or `ComplexPlot3D` plot in Mathematica, it is indeed necessary to specify the range of complex values as a pair of complex numbers. Here's an example code for a 3D surface plot of the Gamma function on the complex plane using the correct syntax:
```mathematica
ComplexPlot3D[Gamma[z], {z, -2 - 2 I, 2 + 2 I}]
```
This creates a 3D surface plot of the Gamma function on the complex plane for the range `-2 - 2 I <= z <= 2 + 2 I`. The resulting plot shows the magnitude of the Gamma function as a surface in 3D space, with contour lines indicating the values of the function.
Thank you for pointing out this important distinction!
Testing the LLM output: Complex plots of the gamma function (2D and 3D).
Like chatGPT, the model can get things wrong, and needs coaching, but it is still impressive to me that you can run this locally on generic hardware (although, this was done on a well specced macbook pro.) I don’t really intend to use this extensively — I was really just playing around. Next I’d like to try on my windows laptop (equipped with a little Nivida 20XX model GPU) and see how that compares to the mac. Is there really a good use case for me to do this on my own hardware — probably not, but it’s really cool!
codemore code
~~~~