Example for Llama2 on Inf2

[namannandan]

Description

This PR adds an example that details the steps to compile and run the Llama2 model on Inferentia2 for text completion with micro batching and response streaming support.

Model: https://huggingface.co/meta-llama/Llama-2-13b-hf
Instance type: inf2.24xlarge

Type of change

• New feature (non-breaking change which adds functionality)

Test

$ torchserve --ncs --start --model-store model_store
$ curl -X POST "http://localhost:8081/models?url=llama-2-13b" 
$ python test_stream_response.py 
Today the weather is really nice and I am planning on going to the beach. I am going to take my camera and take some pictures. I am going to take pictures of the beach and the ocean. I am going to 
take pictures of the people and the animals. I am going to take pictures of the sun and the sky. I am going to take pictures of the sand and the water. I am going to take pictures of the waves and 
the birds. I am going to take pictures of the shells and the rocks. I am going to

[codecov]

Codecov Report

Merging #2458 (bbcdaf2) into master (80b1679) will decrease coverage by 0.59%.
The diff coverage is 12.12%.

❗ Current head bbcdaf2 differs from pull request most recent head cfaf385. Consider uploading reports for the commit cfaf385 to get more accurate results

@@            Coverage Diff             @@
##           master    #2458      +/-   ##
==========================================
- Coverage   70.87%   70.29%   -0.59%     
==========================================
  Files          83       84       +1     
  Lines        3839     3871      +32     
  Branches       58       58              
==========================================
  Hits         2721     2721              
- Misses       1114     1146      +32     
  Partials        4        4              

Files Changed	Coverage Δ
ts/handler_utils/hf_batch_streamer.py	0.00% <0.00%> (ø)
ts/handler_utils/micro_batching.py	90.29% <80.00%> (-0.62%)	⬇️

... and 1 file with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[lxning]

save_split_checkpoints.py is common. Please move it to large_models/util/ and rename it as inf2_save_split_checkpoints.py.

[lxning]

Please add feature AOT precompile and store model in cache in this example

• update README
• replace inf2_handler.py wtih the new one

[namannandan]

Issue to track follow up tasks on this PR: #2600

[HamidShojanazeri]

@namannandan I am wondering if compilation can be done a head of time and we just load the compiled graphs here the way it was working for inf1?

[lxning]

I tested the _save_compiled_artifacts . It is able to generate a neuron model. However, the transformers_neuronx still needs to recompile. I already let Neuron team know they need more work on the experimental feature _save_compiled_artifacts.

Addressed review comments. Follow up tasks tracked here: #2600