Resnet18 torch.compile example

[SimonTong22]

Description

Adding an example for TorchServe to use torch.compile for a Resnet18 model, similar to what already exists for densenet161.

Type of change

• New feature (non-breaking change which adds functionality)

Feature/Issue validation/testing

TorchServe start logs from running:

torchserve --start --model-store model_store --models resnet-18=resnet-18.mar --ts-config config.properties

2024-05-02T16:15:39,305 [INFO ] main org.pytorch.serve.ModelServer - Loading initial models: resnet-18.mar
2024-05-02T16:15:40,423 [INFO ] main org.pytorch.serve.wlm.ModelManager - Model resnet-18 loaded.
2024-05-02T16:15:40,551 [INFO ] main org.pytorch.serve.ModelServer - Initialize Inference server with: EpollServerSocketChannel.
2024-05-02T16:15:41,438 [INFO ] main org.pytorch.serve.ModelServer - Inference API bind to: http://localhost:8443
2024-05-02T16:15:41,438 [INFO ] main org.pytorch.serve.ModelServer - Initialize Management server with: EpollServerSocketChannel.
2024-05-02T16:15:41,486 [INFO ] main org.pytorch.serve.ModelServer - Management API bind to: http://localhost:8081
2024-05-02T16:15:41,487 [INFO ] main org.pytorch.serve.ModelServer - Initialize Metrics server with: EpollServerSocketChannel.
2024-05-02T16:15:41,488 [INFO ] main org.pytorch.serve.ModelServer - Metrics API bind to: http://localhost:8082
2024-05-02T16:15:43,545 [INFO ] W-9026-resnet-18_1.0-stdout MODEL_LOG - Torch worker started.
2024-05-02T16:15:43,589 [INFO ] W-9026-resnet-18_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9026.
2024-05-02T16:15:48,600 [INFO ] W-9016-resnet-18_1.0-stdout MODEL_LOG - Compiled model with backend inductor, mode reduce-overhead

Inference logs from running:

curl http://localhost:8443/predictions/resnet-18 -T ../../image_classifier/kitten.jpg

2024-05-02T16:21:13,176 [INFO ] W-9016-resnet-18_1.0-stdout MODEL_LOG - Backend received inference at: 1714692073
2024-05-02T16:21:29,707 [INFO ] W-9016-resnet-18_1.0-stdout MODEL_METRICS - HandlerTime.ms:16527.88|#ModelName:resnet-18,Level:Model|#hostname:devvm6604.frc0.facebook.com,requestID:f1227f27-4c97-40e7-be39-950c42db64be,timestamp:1714692089
2024-05-02T16:21:29,707 [INFO ] W-9016-resnet-18_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - result=[METRICS]PredictionTime.Milliseconds:16528.77|#ModelName:resnet-18,Level:Model|#type:GAUGE|#hostname:devvm6604.frc0.facebook.com,1714692089,f1227f27-4c97-40e7-be39-950c42db64be, pattern=[METRICS]
2024-05-02T16:21:29,707 [INFO ] W-9016-resnet-18_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - result=[METRICS]PredictionTime.Milliseconds:16528.77|#ModelName:resnet-18,Level:Model|#type:GAUGE|#hostname:devvm6604.frc0.facebook.com,1714692089,f1227f27-4c97-40e7-be39-950c42db64be, pattern=[METRICS]
2024-05-02T16:21:29,707 [INFO ] W-9016-resnet-18_1.0-stdout MODEL_METRICS - PredictionTime.ms:16528.77|#ModelName:resnet-18,Level:Model|#hostname:devvm6604.frc0.facebook.com,requestID:f1227f27-4c97-40e7-be39-950c42db64be,timestamp:1714692089
2024-05-02T16:21:29,709 [INFO ] W-9016-resnet-18_1.0 ACCESS_LOG - /127.0.0.1:55302 "PUT /predictions/resnet-18 HTTP/1.1" 200 16540
2024-05-02T16:21:29,710 [INFO ] W-9016-resnet-18_1.0 TS_METRICS - Requests2XX.Count:1.0|#Level:Host|#hostname:devvm6604.frc0.facebook.com,timestamp:1714692089
2024-05-02T16:21:29,710 [INFO ] W-9016-resnet-18_1.0 TS_METRICS - ts_inference_latency_microseconds.Microseconds:1.6535149711E7|#model_name:resnet-18,model_version:default|#hostname:devvm6604.frc0.facebook.com,timestamp:1714692089
2024-05-02T16:21:29,710 [INFO ] W-9016-resnet-18_1.0 TS_METRICS - ts_queue_latency_microseconds.Microseconds:269.416|#model_name:resnet-18,model_version:default|#hostname:devvm6604.frc0.facebook.com,timestamp:1714692089

Client side:

{
  "tabby": 0.40966343879699707,
  "tiger_cat": 0.346704363822937,
  "Egyptian_cat": 0.13002890348434448,
  "lynx": 0.023919545114040375,
  "bucket": 0.011532172560691833
}

Checklist:

• Did you have fun?
• Have you added tests that prove your fix is effective or that this feature works?
• Has code been commented, particularly in hard-to-understand areas?
• Have you made corresponding changes to the documentation?

[agunapal]

LGTM