components model_prediction_with_container

Distributed Model Prediction

model_prediction_with_container

Overview

Optimized Distributed inference component for LLMs.

Version: 0.0.7

Tags

type : evaluation sub_type : subgraph Preview

View in Studio: https://ml.azure.com/registries/azureml/components/model_prediction_with_container/version/0.0.7

Inputs

model prediction

Name	Description	Type	Default	Optional	Enum
task	Task type	string	text-classification		['text-classification', 'text-classification-multilabel', 'text-named-entity-recognition', 'text-summarization', 'question-answering', 'text-translation', 'text-generation', 'fill-mask', 'chat-completion']
test_data	Test Data	uri_folder		False
mlflow_model	Mlflow model	mlflow_model		False
label_column_name	Label column name in provided test dataset (Ex: label)	string
input_column_names	Input column names in provided test dataset (Ex : column1). Add comma delimited values in case of multiple input columns (Ex : column1,column2)	string		True
mini_batch_size		integer	100	True
evaluation_config_params	JSON Serialized string of evaluation_config	string		True

Outputs

Name	Description	Type
predictions		uri_file
ground_truth		uri_file
prediction_probabilities		uri_file

TODO: Not implemented yet in this component

Name	Description	Type
performance_metadata	Json File containing metadata for perf output	uri_file

Environment

azureml://registries/azureml/environments/foundation-model-inference/labels/latest