xinference.client.handlers.GenerateModelHandle.generate

GenerateModelHandle.generate(prompt: str, generate_config: PytorchGenerateConfig | None = None) → Completion | Iterator[CompletionChunk]

Creates a completion for the provided prompt and parameters via RESTful APIs.

Parámetros:

• prompt (str) – The user’s message or user’s input.

• generate_config (Optional["PytorchGenerateConfig"]) – Additional configuration for the chat generation. «PytorchGenerateConfig» -> Configuration for pytorch model

Devuelve:

Stream is a parameter in generate_config. When stream is set to True, the function will return Iterator[«CompletionChunk»]. When stream is set to False, the function will return «Completion».

Tipo del valor devuelto:

Union[«Completion», Iterator[«CompletionChunk»]]

Muestra:

RuntimeError – Fail to generate the completion from the server. Detailed information provided in error message.