NaviGator Toolkit

NaviGator Toolkit is an OpenAI-compatible tool that provides API access to large language models such as GPT, Llama, Gemini, and Claude. Image generation models like DALL-E and speech-to-text models like Whisper are also available.

NaviGator Toolkit allows every UF student, faculty, and staff to generate API keys and leverage AI models in their applications via the API. Each user has a $5 monthly budget that they can spend on locally deployed models such as Llama, Mixtral, Gemma, Codestral, Stable Diffusion, and Mistral. Additionally, researchers can be onboarded as teams with custom budgets to leverage cloud models. Researchers are encouraged to submit a request via the UFIT Help Portal.

Info

While this page provides some documentation, and the LLMs are running on HiPerGator hardware, please use the UFIT Help Portal for support for NaviGator Toolkit, not the UFIT Research Computing portal.

Generating an API key

1. Visit the NaviGator Toolkit page to view information about NaviGator Toolkit.
2. Scroll down the instructions to create your API key and create your key.
3. Remember to copy your key before closing the window.
4. Also note that keys expire after 1-year (or when you manually expire them)

Never put your key in a script!

Your API key is what controls access to your NaviGator Toolkit account. While it is possible to paste the key into a script or notebook as clear text, that is poor security practice!

At a minimum, API keys should be stored in a separate file, and read into scripts. If you use Git, that file should not be in the repository folder as it may accidentally be added and committed.

Make a .json file on HiPerGator to store you key securely.

There are many methods of managing your key. The method outlined here uses a .json file that is read into your scripts.

1. It is suggested to put your API key in a file in your home directory.

2. Make a text file with the following contents:

   {
   "OPENAI_API_KEY" : "Put your key here in the quotes",
   "base_url" : "https://api.ai.it.ufl.edu/"
   }
   

   Save the file so you remember what it is, for example, navigator_api_keys.json

   Note that the base_url, doesn't need to be part of this, but it keeps all the local configuration in one place.

Example code

The NaviGator_Toolkit_API GitHub repository has sample Jupyter Notebooks that you can use as examples or to test things.

This script compiles the cells from NaviGator_test.ipynb

NaviGator_test.py

import openai
import os
import json

# Set the path to your jsnkey file
key_file = '/home/magitz/navigator_api_keys.json'

# Load the JSON file
with open(key_file, 'r') as file:
    data = json.load(file)

# Extract the values
OPENAI_API_KEY = data.get('OPENAI_API_KEY')
base_url = data.get('base_url')

# Set the environment variable
os.environ['TOOLKIT_API_KEY'] = OPENAI_API_KEY

# Check list of available models
client = openai.OpenAI(
    api_key=os.environ.get("TOOLKIT_API_KEY"),
    base_url=base_url
)

response = client.models.list()

print(response)

# Set model from list above
model = "llama-3.1-70b-instruct"

response = client.chat.completions.create(
    model=model,
    messages = [
        {
            "role": "user",
            "content": "Write a short poem about somethign fun."
        }
    ]
)

print(response)

# Print nicely formatted message
print(response.choices[0].message.content)

NaviGator uses the OpenAI API. Please see OpenAI's documentation for additional functionality.