ResNet models tuning with NVIDIA AITune
This example demonstrates how to use NVIDIA AITune to tune a ResNet model.
Environment Setup
You can use either of the following options to set up the environment:
Option 1 - virtual environment managed by you
Activate your virtual environment and install the dependencies:
Option 2 - virtual environment managed by uv
Install dependencies:
Usage
To tune the ResNet model, run:
or for uv:
After tuning, run classification with:
or for uv:
AI Dynamo ResNet Deployment with Batching
To run ResNet as AI Dynamo service with dynamic batching, we have prepared additional configs and scripts.
Code starts in resnet/dynamo/backend.py, Docker and Docker Compose is used to make setup simple.
Firstly, start all services by running docker compose --profile all up --detach. This will build and start all required services.
After successful tuning and services start run below commands to test the service.
Single Request
python -m resnet.dynamo.client --num-requests 1
# response should be:
# {"request_id":"img-resnet-single","prediction":"golden retriever","confidence":0.9992710947990417,"class_id":207,"inference_time":0.123}
Batching Demonstration
python -m resnet.dynamo.client --num-requests 2
python -m resnet.dynamo.client --num-requests 4
python -m resnet.dynamo.client --num-requests 8
python -m resnet.dynamo.client --num-requests 100
Custom Image
Finally, to shut it down use docker compose down --volumes.
Dynamic batching
The service uses dynamic batching — requests are grouped and processed together for efficiency. Currently, there is one frontend and one worker. To support multiple workers, move batching to a separate service that handles request grouping.
Model Details
Can be found in following pages: * https://pytorch.org/vision/stable/models.html#classification * https://huggingface.co/timm