generative-ts

Function createGroqModelProvider

  • createGroqModelProvider<THttpClientOptions>(params): HttpModelProvider<OpenAiChatOptions, OpenAiChatResponse, THttpClientOptions, {
        modelId: string;
    }>

  • Creates a Groq ModelProvider with the OpenAiChatApi

    import { createGroqModelProvider } from "generative-ts";

    const llama3 = createGroqModelProvider({
      modelId: "llama3-70b-8192", // Groq defined model ID
      // you can explicitly pass auth here, otherwise by default it is read from process.env
    });

    const response = await llama3.sendRequest({
      $prompt: "Brief History of NY Mets:"
      // all other OpenAI ChatCompletion options available here (Groq uses the OpenAI ChatCompletion API for all the models it hosts)
    });

    console.log(response.choices[0]?.message.content);

    Provider Setup and Notes

    Create an API account at Groq

    Obtain a Groq API key and either pass them explicitly in auth or set them in the environment as GROQ_API_KEY

    Groq uses the OpenAI ChatCompletion API for all the models it hosts.

    Model Parameters

    Model IDs

    Type Parameters

    • THttpClientOptions = HttpClientOptions

    Parameters

    • params: {
          auth?: GroqAuthConfig;
          client?: HttpClient<THttpClientOptions>;
          modelId: string;
      }
      • Optional auth?: GroqAuthConfig

        Authentication configuration for Groq. If not supplied, it will be loaded from the environment.

      • Optional client?: HttpClient<THttpClientOptions>

        HTTP client to use for requests. If not supplied, the built-in fetch-based implementation will be used.

      • modelId: string

        The model ID as defined by Groq

    Returns HttpModelProvider<OpenAiChatOptions, OpenAiChatResponse, THttpClientOptions, {
        modelId: string;
    }>

    The Groq Model Provider with the OpenAiChatApi

    See

    Throws

    If no auth is passed and GROQ_API_KEY is not found in process.env

    Example: Usage

    import { createGroqModelProvider } from "generative-ts";

    const llama3 = createGroqModelProvider({
      modelId: "llama3-70b-8192",
    });

    const response = await llama3.sendRequest({ $prompt: "Brief History of NY Mets:" });

    console.log(response.choices[0]?.message.content);

On This Page

Provider Setup and NotesModel ParametersModel IDs