Refactor conda build script to publish all binaries from ubuntu machine

[agunapal]

Description

This PR refactor's conda build scripts to
- Build conda binaries for linux
- Use conda convert to create binaries for all other platforms
- Currently, the platforms are win-64 and osx-64
- To publish to a new platform, ex: osx-arm64, we just need to add this string to the list
- Changes torchserve binary publishing workflows to use only ubuntu

Fixes #(issue)

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• New feature (non-breaking change which adds functionality)
• This change requires a documentation update

Feature/Issue validation/testing

Below is a screenshot of files being uploaded from a workflow

You can find all the files here
https://anaconda.org/agunapal/repo

Workflow run is here
https://github.com/agunapal/serve/actions/runs/6032298050

• Regression Tests

Installing Binaries on osx-64 with Python 3.10

(py310) ec2-user@ip-172-31-24-146 serve % conda install -c agunapal torchserve torch-model-archiver torch-workflow-archiver
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /Users/ec2-user/anaconda3/envs/py310

  added / updated specs:
    - torch-model-archiver
    - torch-workflow-archiver
    - torchserve


The following NEW packages will be INSTALLED:

  freetype           pkgs/main/osx-64::freetype-2.12.1-hd8bbffd_0 
  giflib             pkgs/main/osx-64::giflib-5.2.1-h6c40b1e_3 
  jpeg               pkgs/main/osx-64::jpeg-9e-h6c40b1e_1 
  lcms2              pkgs/main/osx-64::lcms2-2.12-hf1fd2bf_0 
  lerc               pkgs/main/osx-64::lerc-3.0-he9d5cce_0 
  libdeflate         pkgs/main/osx-64::libdeflate-1.17-hb664fd8_0 
  libpng             pkgs/main/osx-64::libpng-1.6.39-h6c40b1e_0 
  libtiff            pkgs/main/osx-64::libtiff-4.5.0-hcec6c5f_2 
  libwebp            pkgs/main/osx-64::libwebp-1.2.4-hf6ce154_1 
  libwebp-base       pkgs/main/osx-64::libwebp-base-1.2.4-h6c40b1e_1 
  lz4-c              pkgs/main/osx-64::lz4-c-1.9.4-hcec6c5f_0 
  pillow             pkgs/main/osx-64::pillow-9.4.0-py310hcec6c5f_0 
  psutil             pkgs/main/osx-64::psutil-5.9.0-py310hca72f7f_0 
  torch-model-archi~ agunapal/osx-64::torch-model-archiver-0.8.2.dev20230831-py310_0 
  torch-workflow-ar~ agunapal/osx-64::torch-workflow-archiver-0.2.10.dev20230831-py310_0 
  torchserve         agunapal/osx-64::torchserve-0.8.2.dev20230831-py310_0 
  zstd               pkgs/main/osx-64::zstd-1.5.5-hc035e20_0 


Proceed ([y]/n)? y


Downloading and Extracting Packages

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
(py310) ec2-user@ip-172-31-24-146 serve % pip list | grep -i torch                                                         
torch                   2.0.1
torch-model-archiver    0.8.2b20230831
torch-workflow-archiver 0.2.10b20230831
torchaudio              2.0.2
torchdata               0.6.1
torchpippy              0.1.1
torchserve              0.8.2b20230831
torchtext               0.15.2
torchvision             0.15.2

Test

I commented install_from_src in our regression scripts and ran the following

## In directory: /Users/ec2-user/serve/test/pytest | Executing command: python -m pytest -v ./
========================================================================= test session starts ==========================================================================
platform darwin -- Python 3.10.0, pytest-7.3.1, pluggy-1.3.0 -- /Users/ec2-user/anaconda3/envs/py310/bin/python
cachedir: .pytest_cache
rootdir: /Users/ec2-user/serve
plugins: mock-3.10.0, cov-4.1.0
collected 109 items                                                                                                                                                    

test_distributed_inference_handler.py::test_large_model_inference SKIPPED (Distributed inference requires multi-gpu machine, skipping for now)                   [  0%]
test_example_dcgan.py::test_model_archive_creation PASSED                                                                                                        [  1%]
test_example_dcgan.py::test_model_register_unregister PASSED                                                                                                     [  2%]
test_example_dcgan.py::test_image_generation_without_any_input_constraints PASSED                                                                                [  3%]
test_example_dcgan.py::test_image_generation_with_input_constraints PASSED                                                                                       [  4%]
test_example_intel_extension_for_pytorch.py::test_single_worker_affinity SKIPPED (Make sure intel-extension-for-pytorch is installed and torch.backends.xeon...) [  5%]
test_example_intel_extension_for_pytorch.py::test_multi_worker_affinity SKIPPED (Make sure intel-extension-for-pytorch is installed and torch.backends.xeon....) [  6%]
test_example_intel_extension_for_pytorch.py::test_worker_scale_up_affinity SKIPPED (Make sure intel-extension-for-pytorch is installed and torch.backends.xe...) [  7%]
test_example_intel_extension_for_pytorch.py::test_worker_scale_down_affinity SKIPPED (Make sure intel-extension-for-pytorch is installed and torch.backends....) [  8%]
test_example_micro_batching.py::test_single_example_inference[yaml_config] PASSED                                                                                [  9%]
test_example_micro_batching.py::test_multi_example_inference[4-yaml_config] PASSED                                                                               [ 10%]
test_example_micro_batching.py::test_multi_example_inference[4-no_config] PASSED                                                                                 [ 11%]
test_example_micro_batching.py::test_single_example_inference[no_config] 2023-08-31T19:17:04,282 [INFO ] W-9001-image_classifier_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 35
2023-08-31T19:17:04,282 [INFO ] W-9001-image_classifier_1.0 TS_METRICS - WorkerThreadTime.Milliseconds:3.0|#Level:Host|#hostname:ip-172-31-24-146.us-west-2.compute.internal,timestamp:1693509424
PASSED                                                                                  [ 11%]
test_example_micro_batching.py::test_multi_example_inference[16-no_config] PASSED                                                                                [ 12%]
test_example_micro_batching.py::test_multi_example_inference[16-yaml_config] PASSED                                                                              [ 13%]
test_example_scriptable_tokenzier.py::test_handler PASSED                                                                                                        [ 14%]
test_example_scriptable_tokenzier.py::test_inference_with_untrained_model_and_sample_text PASSED                                                                 [ 15%]
test_example_scriptable_tokenzier.py::test_inference_with_untrained_model_and_empty_string 2023-08-31T19:18:09,575 [INFO ] W-9001-scriptable_tokenizer_untrained_1.0 TS_METRICS - QueueTime.Milliseconds:0.0|#Level:Host|#hostname:ip-172-31-24-146.us-west-2.compute.internal,timestamp:1693509489
2023-08-31T19:18:09,575 [DEBUG] W-9001-scriptable_tokenizer_untrained_1.0 org.pytorch.serve.wlm.WorkerThread - sent a reply, jobdone: true
2023-08-31T19:18:09,575 [INFO ] W-9001-scriptable_tokenizer_untrained_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 311
2023-08-31T19:18:09,575 [INFO ] W-9001-scriptable_tokenizer_untrained_1.0 TS_METRICS - WorkerThreadTime.Milliseconds:2.0|#Level:Host|#hostname:ip-172-31-24-146.us-west-2.compute.internal,timestamp:1693509489
PASSED                                                                [ 16%]
test_example_scriptable_tokenzier.py::test_inference_with_pretrained_model PASSED                                                                                [ 17%]
test_gRPC_inference_api.py::test_inference_apis PASSED                                                                                                           [ 18%]
test_gRPC_inference_api.py::test_inference_stream_apis PASSED                                                                                                    [ 19%]
test_gRPC_management_apis.py::test_management_apis PASSED                                                                                                        [ 20%]
test_handler.py::test_mnist_model_register_and_inference_on_valid_model PASSED                                                                                   [ 21%]
test_handler.py::test_mnist_model_register_using_non_existent_handler_with_nonzero_workers PASSED                                                                [ 22%]
test_handler.py::test_mnist_model_register_scale_inference_with_non_existent_handler 2023-08-31T19:21:36,648 [WARN ] W-9003-mnist_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9003-mnist_1.0-stderr
2023-08-31T19:21:36,649 [INFO ] W-9002-mnist_1.0-stdout MODEL_LOG - [PID]1318
2023-08-31T19:21:36,648 [INFO ] W-9003-mnist_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9003-mnist_1.0-stderr
2023-08-31T19:21:36,649 [WARN ] W-9003-mnist_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9003-mnist_1.0-stdout
2023-08-31T19:21:36,649 [INFO ] W-9002-mnist_1.0-stdout MODEL_LOG - Torch worker started.
2023-08-31T19:21:36,649 [DEBUG] W-9002-mnist_1.0 org.pytorch.serve.wlm.WorkerThread - W-9002-mnist_1.0 State change WORKER_STOPPED -> WORKER_STARTED
2023-08-31T19:21:36,649 [INFO ] W-9003-mnist_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9003 in 1 seconds.
2023-08-31T19:21:36,649 [INFO ] W-9002-mnist_1.0-stdout MODEL_LOG - Python runtime: 3.10.0
2023-08-31T19:21:36,649 [INFO ] W-9002-mnist_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /var/folders/xn/m8y264514m5gzfw5_60chh2w0000gn/T//.ts.sock.9002
PASSED                                                                      [ 22%]2023-08-31T19:21:36,650 [DEBUG] W-9002-mnist_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED
2023-08-31T19:21:36,650 [DEBUG] W-9002-mnist_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died.

java.lang.InterruptedException: null
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1679) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:515) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:677) ~[?:?]
at org.pytorch.serve.wlm.Model.pollBatch(Model.java:276) ~[model-server.jar:?]
at org.pytorch.serve.wlm.BatchAggregator.getRequest(BatchAggregator.java:34) ~[model-server.jar:?]
at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:186) [model-server.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
at java.lang.Thread.run(Thread.java:833) [?:?]
test_handler.py::test_mnist_model_register_and_inference_on_valid_model_explain 2023-08-31T19:21:36,650 [DEBUG] W-9002-mnist_1.0 org.pytorch.serve.wlm.WorkerThread - W-9002-mnist_1.0 State change WORKER_STARTED -> WORKER_STOPPED
2023-08-31T19:21:36,650 [WARN ] W-9002-mnist_1.0 org.pytorch.serve.wlm.WorkerThread - Auto recovery failed again
2023-08-31T19:21:36,650 [INFO ] W-9002-mnist_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9002-mnist_1.0-stdout
2023-08-31T19:21:36,650 [WARN ] W-9002-mnist_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9002-mnist_1.0-stderr
2023-08-31T19:21:36,650 [WARN ] W-9002-mnist_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9002-mnist_1.0-stdout
2023-08-31T19:21:36,650 [INFO ] W-9002-mnist_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9002-mnist_1.0-stderr
2023-08-31T19:21:36,650 [INFO ] W-9002-mnist_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9002 in 1 seconds.
PASSED                                                                           [ 23%]
test_handler.py::test_kserve_mnist_model_register_and_inference_on_valid_model PASSED                                                                            [ 24%]
test_handler.py::test_kserve_mnist_model_register_scale_inference_with_non_existent_handler 2023-08-31T19:21:50,436 [INFO ] W-9001-mnist_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9001-mnist_1.0-stdout
2023-08-31T19:21:50,436 [WARN ] W-9001-mnist_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9001-mnist_1.0-stderr
2023-08-31T19:21:50,436 [WARN ] W-9001-mnist_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9001-mnist_1.0-stdout
2023-08-31T19:21:50,436 [INFO ] W-9001-mnist_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9001-mnist_1.0-stderr
2023-08-31T19:21:50,436 [INFO ] W-9001-mnist_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9001 in 1 seconds.
2023-08-31T19:21:50,436 [INFO ] W-9002-mnist_1.0-stdout MODEL_LOG - Successfully loaded /Users/ec2-user/anaconda3/envs/py310/lib/python3.10/site-packages/ts/configs/metrics.yaml.
2023-08-31T19:21:50,436 [INFO ] W-9002-mnist_1.0-stdout MODEL_LOG - [PID]1340
PASSED                                                               [ 25%]2023-08-31T19:21:50,437 [INFO ] W-9002-mnist_1.0-stdout MODEL_LOG - Torch worker started.
2023-08-31T19:21:50,437 [DEBUG] W-9002-mnist_1.0 org.pytorch.serve.wlm.WorkerThread - W-9002-mnist_1.0 State change WORKER_STOPPED -> WORKER_STARTED
2023-08-31T19:21:50,437 [INFO ] W-9002-mnist_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /var/folders/xn/m8y264514m5gzfw5_60chh2w0000gn/T//.ts.sock.9002
2023-08-31T19:21:50,437 [INFO ] W-9002-mnist_1.0-stdout MODEL_LOG - Python runtime: 3.10.0

test_handler.py::test_kserve_mnist_model_register_and_inference_on_valid_model_explain 2023-08-31T19:21:50,437 [INFO ] KQueueEventLoopGroup-5-3 org.pytorch.serve.wlm.WorkerThread - 9002 Worker disconnected. WORKER_STARTED
2023-08-31T19:21:50,437 [INFO ] W-9002-mnist_1.0-stdout MODEL_LOG - Connection accepted: /var/folders/xn/m8y264514m5gzfw5_60chh2w0000gn/T//.ts.sock.9002.
2023-08-31T19:21:50,437 [DEBUG] W-9002-mnist_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED
2023-08-31T19:21:50,438 [DEBUG] W-9002-mnist_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died.
java.lang.InterruptedException: null
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1679) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:515) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:677) ~[?:?]
at org.pytorch.serve.wlm.Model.pollBatch(Model.java:276) ~[model-server.jar:?]
at org.pytorch.serve.wlm.BatchAggregator.getRequest(BatchAggregator.java:34) ~[model-server.jar:?]
PASSED                                                                    [ 26%]
test_handler.py::test_huggingface_bert_batch_inference PASSED                                                                                                    [ 27%]
test_handler.py::test_MMF_activity_recognition_model_register_and_inference_on_valid_model SKIPPED (MMF doesn't support PT 1.10 yet)                             [ 28%]
test_handler.py::test_huggingface_bert_model_parallel_inference PASSED                                                                                           [ 29%]
test_handler.py::test_echo_stream_inference PASSED                                                                                                               [ 30%]
test_metrics.py::test_logs_created PASSED                                                                                                                        [ 31%]
test_metrics.py::test_logs_startup_cfg_created_snapshot_enabled PASSED                                                                                           [ 32%]
test_metrics.py::test_logs_startup_cfg_created_snapshot_disabled PASSED                                                                                          [ 33%]
test_metrics.py::test_metrics_startup_cfg_created_snapshot_enabled PASSED                                                                                        [ 33%]
test_metrics.py::test_metrics_startup_cfg_created_snapshot_disabled PASSED                                                                                       [ 34%]
test_metrics.py::test_log_location_var_snapshot_disabled PASSED                                                                                                  [ 35%]
test_metrics.py::test_log_location_var_snapshot_enabled PASSED                                                                                                   [ 36%]
test_metrics.py::test_async_logging PASSED                                                                                                                       [ 37%]
test_metrics.py::test_async_logging_non_boolean PASSED                                                                                                           [ 38%]
test_metrics.py::test_metrics_location_var_snapshot_disabled PASSED                                                                                              [ 39%]
test_metrics.py::test_metrics_location_var_snapshot_enabled PASSED                                                                                               [ 40%]
test_metrics.py::test_log_location_and_metric_location_vars_snapshot_enabled PASSED                                                                              [ 41%]
test_metrics.py::test_log_location_var_snapshot_disabled_custom_path_read_only PASSED                                                                            [ 42%]
test_metrics.py::test_metrics_location_var_snapshot_enabled_rdonly_dir PASSED                                                                                    [ 43%]
test_metrics.py::test_metrics_log_mode PASSED                                                                                                                    [ 44%]
test_metrics.py::test_metrics_prometheus_mode PASSED                                                                                                             [ 44%]
test_metrics.py::test_collect_system_metrics_when_not_disabled PASSED                                                                                            [ 45%]
test_metrics.py::test_disable_system_metrics_using_config_properties PASSED                                                                                      [ 46%]
test_metrics.py::test_disable_system_metrics_using_environment_variable PASSED                                                                                   [ 47%]
test_metrics_kf.py::test_logs_created PASSED                                                                                                                     [ 48%]
test_metrics_kf.py::test_logs_startup_cfg_created_snapshot_enabled PASSED                                                                                        [ 49%]
test_metrics_kf.py::test_logs_startup_cfg_created_snapshot_disabled PASSED                                                                                       [ 50%]
test_metrics_kf.py::test_metrics_startup_cfg_created_snapshot_enabled PASSED                                                                                     [ 51%]
test_metrics_kf.py::test_metrics_startup_cfg_created_snapshot_disabled PASSED                                                                                    [ 52%]
test_metrics_kf.py::test_log_location_var_snapshot_disabled PASSED                                                                                               [ 53%]
test_metrics_kf.py::test_log_location_var_snapshot_enabled PASSED                                                                                                [ 54%]
test_metrics_kf.py::test_async_logging PASSED                                                                                                                    [ 55%]
test_metrics_kf.py::test_async_logging_non_boolean PASSED                                                                                                        [ 55%]
test_metrics_kf.py::test_metrics_location_var_snapshot_disabled PASSED                                                                                           [ 56%]
test_metrics_kf.py::test_metrics_location_var_snapshot_enabled PASSED                                                                                            [ 57%]
test_metrics_kf.py::test_log_location_and_metric_location_vars_snapshot_enabled PASSED                                                                           [ 58%]
test_metrics_kf.py::test_log_location_var_snapshot_disabled_custom_path_read_only PASSED                                                                         [ 59%]
test_metrics_kf.py::test_metrics_location_var_snapshot_enabled_rdonly_dir PASSED                                                                                 [ 60%]
test_model_archiver.py::test_multiple_model_versions_registration PASSED                                                                                         [ 61%]
test_model_archiver.py::test_duplicate_model_registration_using_local_url_followed_by_http_url PASSED                                                            [ 62%]
test_model_archiver.py::test_duplicate_model_registration_using_http_url_followed_by_local_url PASSED                                                            [ 63%]
test_model_archiver.py::test_model_archiver_to_regenerate_model_mar_without_force PASSED                                                                         [ 64%]
test_model_archiver.py::test_model_archiver_to_regenerate_model_mar_with_force PASSED                                                                            [ 65%]
test_model_archiver.py::test_model_archiver_without_handler_flag PASSED                                                                                          [ 66%]
test_model_archiver.py::test_model_archiver_without_model_name_flag PASSED                                                                                       [ 66%]
test_model_archiver.py::test_model_archiver_without_model_file_flag PASSED                                                                                       [ 67%]
test_model_archiver.py::test_model_archiver_without_serialized_flag PASSED                                                                                       [ 68%]
test_onnx.py::test_convert_to_onnx PASSED                                                                                                                        [ 69%]
test_onnx.py::test_model_packaging_and_start PASSED                                                                                                              [ 70%]
test_onnx.py::test_model_start PASSED                                                                                                                            [ 71%]
test_onnx.py::test_inference PASSED                                                                                                                              [ 72%]
test_onnx.py::test_stop PASSED                                                                                                                                   [ 73%]
test_pytorch_profiler.py::test_profiler_default_and_custom_handler[/Users/ec2-user/serve/test/pytest/profiler_utils/resnet_custom.py] PASSED                     [ 74%]
test_pytorch_profiler.py::test_profiler_default_and_custom_handler[image_classifier] PASSED                                                                      [ 75%]
test_pytorch_profiler.py::test_profiler_arguments_override[/Users/ec2-user/serve/test/pytest/profiler_utils/resnet_profiler_override.py] PASSED                  [ 76%]
test_pytorch_profiler.py::test_batch_input[/Users/ec2-user/serve/test/pytest/profiler_utils/resnet_profiler_override.py] PASSED                                  [ 77%]
test_sm_mme_requirements.py::test_no_model_loaded PASSED                                                                                                         [ 77%]
test_sm_mme_requirements.py::test_oom_on_model_load SKIPPED (Test to be run on GPU only)                                                                         [ 78%]2023-08-31T19:27:44,470 [INFO ] pool-3-thread-1 TS_METRICS - CPUUtilization.Percent:0.0|#Level:Host|#hostname:ip-172-31-24-146.us-west-2.compute.internal,timestamp:1693510064

test_sm_mme_requirements.py::test_oom_on_invoke SKIPPED (Test to be run on GPU only)                                                                             [ 79%]2023-08-31T19:27:44,470 [INFO ] pool-3-thread-1 TS_METRICS - DiskAvailable.Gigabytes:63.780609130859375|#Level:Host|#hostname:ip-172-31-24-146.us-west-2.compute.internal,timestamp:1693510064
2023-08-31T19:27:44,471 [INFO ] pool-3-thread-1 TS_METRICS - DiskUsage.Gigabytes:14.271785736083984|#Level:Host|#hostname:ip-172-31-24-146.us-west-2.compute.internal,timestamp:1693510064
2023-08-31T19:27:44,471 [INFO ] pool-3-thread-1 TS_METRICS - DiskUtilization.Percent:18.3|#Level:Host|#hostname:ip-172-31-24-146.us-west-2.compute.internal,timestamp:1693510064
2023-08-31T19:27:44,471 [INFO ] pool-3-thread-1 TS_METRICS - MemoryAvailable.Megabytes:26884.703125|#Level:Host|#hostname:ip-172-31-24-146.us-west-2.compute.internal,timestamp:1693510064
2023-08-31T19:27:44,471 [INFO ] pool-3-thread-1 TS_METRICS - MemoryUsed.Megabytes:5880.30078125|#Level:Host|#hostname:ip-172-31-24-146.us-west-2.compute.internal,timestamp:1693510064

test_snapshot.py::test_snapshot_created_on_start_and_stop PASSED                                                                                                 [ 80%]
test_snapshot.py::test_snapshot_created_on_management_api_invoke PASSED                                                                                          [ 81%]
test_snapshot.py::test_start_from_snapshot PASSED                                                                                                                [ 82%]
test_snapshot.py::test_start_from_latest PASSED                                                                                                                  [ 83%]
test_snapshot.py::test_start_from_read_only_snapshot PASSED                                                                                                      [ 84%]
test_snapshot.py::test_no_config_snapshots_cli_option PASSED                                                                                                     [ 85%]
test_snapshot.py::test_start_from_default PASSED                                                                                                                 [ 86%]
test_snapshot.py::test_start_from_non_existing_snapshot PASSED                                                                                                   [ 87%]
test_snapshot.py::test_torchserve_init_with_non_existent_model_store PASSED                                                                                      [ 88%]
test_snapshot.py::test_restart_torchserve_with_last_snapshot_with_model_mar_removed PASSED                                                                       [ 88%]
test_snapshot.py::test_replace_mar_file_with_dummy PASSED                                                                                                        [ 89%]
test_snapshot.py::test_restart_torchserve_with_one_of_model_mar_removed PASSED                                                                                   [ 90%]
test_torch_compile.py::TestTorchCompile::test_archive_model_artifacts SKIPPED (Skipping test on non-Linux system)                                                [ 91%]
test_torch_compile.py::TestTorchCompile::test_start_torchserve SKIPPED (Skipping test on non-Linux system)                                                       [ 92%]
test_torch_compile.py::TestTorchCompile::test_server_status SKIPPED (Skipping test on non-Linux system)                                                          [ 93%]
test_torch_compile.py::TestTorchCompile::test_registered_model SKIPPED (Skipping test on non-Linux system)                                                       [ 94%]
test_torch_compile.py::TestTorchCompile::test_serve_inference SKIPPED (Skipping test on non-Linux system)                                                        [ 95%]
test_torch_xla.py::TestTorchXLA::test_archive_model_artifacts SKIPPED (PyTorch/XLA is not installed)                                                             [ 96%]
test_torch_xla.py::TestTorchXLA::test_start_torchserve SKIPPED (PyTorch/XLA is not installed)                                                                    [ 97%]
test_torch_xla.py::TestTorchXLA::test_server_status SKIPPED (PyTorch/XLA is not installed)                                                                       [ 98%]
test_torch_xla.py::TestTorchXLA::test_registered_model SKIPPED (PyTorch/XLA is not installed)                                                                    [ 99%]
test_torch_xla.py::TestTorchXLA::test_serve_inference SKIPPED (PyTorch/XLA is not installed)                                                                     [100%]

=========================================================================== warnings summary ===========================================================================
test_torch_compile.py:11
  /Users/ec2-user/serve/test/pytest/test_torch_compile.py:11: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    from pkg_resources import packaging

../../../anaconda3/envs/py310/lib/python3.10/site-packages/pkg_resources/__init__.py:2871
  /Users/ec2-user/anaconda3/envs/py310/lib/python3.10/site-packages/pkg_resources/__init__.py:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('mpl_toolkits')`.
  Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
    declare_namespace(pkg)

test/pytest/test_example_scriptable_tokenzier.py::test_handler
  /Users/ec2-user/anaconda3/envs/py310/lib/python3.10/site-packages/torch/jit/_recursive.py:266: UserWarning: 'batch_first' was found in ScriptModule constants, but was not actually set in __init__. Consider removing it.
    warnings.warn("'{}' was found in ScriptModule constants, "

test/pytest/test_example_scriptable_tokenzier.py::test_handler
  /Users/ec2-user/anaconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py:1501: UserWarning: The PyTorch API of nested tensors is in prototype stage and will change in the near future. (Triggered internally at /Users/runner/work/pytorch/pytorch/pytorch/aten/src/ATen/NestedTensorImpl.cpp:179.)
    return forward_call(*args, **kwargs)

test/pytest/test_example_scriptable_tokenzier.py::test_handler
  /Users/ec2-user/serve/test/pytest/../../examples/text_classification_with_scriptable_tokenizer/handler.py:97: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
    data = F.softmax(data)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
======================================================== 91 passed, 18 skipped, 5 warnings in 821.28s (0:13:41) ========================================================
Removing file : inference_pb2_grpc.py
Removing file : inference_pb2.py
Removing file : management_pb2.py
Removing file : management_pb2_grpc.py
## Deleting model_store_gen_dir: /Users/ec2-user/serve/ts_scripts/../model_store_gen

(py310) ec2-user@ip-172-31-24-146 serve % 

Checklist:

• Did you have fun?
• Have you added tests that prove your fix is effective or that this feature works?
• Has code been commented, particularly in hard-to-understand areas?
• Have you made corresponding changes to the documentation?

[msaroufim]

Could you also please sanity check that a converted binary also works? Most convincing would be to run our test suite

[codecov]

Codecov Report

Merging #2561 (857a8fe) into master (242895c) will not change coverage.
The diff coverage is n/a.

❗ Current head 857a8fe differs from pull request most recent head 0f08a62. Consider uploading reports for the commit 0f08a62 to get more accurate results

@@           Coverage Diff           @@
##           master    #2561   +/-   ##
=======================================
  Coverage   72.71%   72.71%           
=======================================
  Files          79       79           
  Lines        3742     3742           
  Branches       58       58           
=======================================
  Hits         2721     2721           
  Misses       1017     1017           
  Partials        4        4           

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[agunapal]

@msaroufim Added logs of the manual test

[msaroufim]

few minor nits left otherwise lgtm