LLM Analysis¶
Latency and memory analysis of Large Language Models (LLMs) / Transformer Models for training and inference.
Analysis¶
Configurations¶
- Configurations
DtypeConfig
EnhancedJSONEncoder
GPUConfig
ModelConfig
ParallelismConfig
dump_configs()
dump_hf_model_configs_by_type_and_task()
dump_model_config_by_name()
get_dtype_config_by_name()
get_gpu_config_by_name()
get_hf_models_by_type_and_task()
get_model_config_by_name()
get_model_config_from_hf()
list_dtype_configs()
list_gpu_configs()
list_model_configs()
populate_model_and_gpu_configs()
read_configs()