Added test cases for SageMaker MME requirements

[agunapal]

Description

Added 2 test cases for SageMaker MME requirements

• Model not loaded (404)
• OOM Error (507

APIs defined here

Fixes #(issue)

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• New feature (non-breaking change which adds functionality)
• This change requires a documentation update

Feature/Issue validation/testing

Please describe the Unit or Integration tests that you ran to verify your changes and relevant result summary. Provide instructions so it can be reproduced.
Please also list any relevant details for your test configuration.

• PyTest

python -m pytest -v test_sm_mme_requirements.py 
============================================================================= test session starts =============================================================================
platform linux -- Python 3.8.16, pytest-7.2.2, pluggy-1.0.0 -- /home/ubuntu/anaconda3/envs/torchserve/bin/python
cachedir: .pytest_cache
rootdir: /home/ubuntu/serve
plugins: cov-4.0.0, mock-3.10.0
collected 3 items                                                                                                                                                             

test_sm_mme_requirements.py::test_no_model_loaded PASSED                 [ 33%]
test_sm_mme_requirements.py::test_oom_on_model_load PASSED               [ 66%]
test_sm_mme_requirements.py::test_oom_on_invoke PASSED                   [100%]

• Regression
  • 1_regression-gpu (cu117).txt
  • 2_regression-gpu (cu118).txt
  • 1_regression-cpu (ubuntu-20.04).txt
  • 2_regression-cpu (macOS-latest).txt

Checklist:

• Did you have fun?
• Have you added tests that prove your fix is effective or that this feature works?
• Has code been commented, particularly in hard-to-understand areas?
• Have you made corresponding changes to the documentation?

[codecov]

Codecov Report

Merging #2200 (3e86d7f) into master (98ca286) will decrease coverage by 0.16%.
The diff coverage is 0.00%.

❗ Current head 3e86d7f differs from pull request most recent head 967a5f7. Consider uploading reports for the commit 967a5f7 to get more accurate results

@@            Coverage Diff             @@
##           master    #2200      +/-   ##
==========================================
- Coverage   71.52%   71.37%   -0.16%     
==========================================
  Files          73       73              
  Lines        3329     3336       +7     
  Branches       57       57              
==========================================
  Hits         2381     2381              
- Misses        945      952       +7     
  Partials        3        3              

Impacted Files	Coverage Δ
ts/model_service_worker.py	65.71% <0.00%> (-1.94%)	⬇️
ts/service.py	71.60% <0.00%> (-2.76%)	⬇️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[agunapal]

@lxning

Now on model loading, I am getting code 507,

but the message is as follows

{
  "code": 507,
  "type": "InternalServerException",
  "message": "Failed to start workers for model BERTSeqClassification version: 1.0"
}

This happens because here we are setting this. I don't know if we want to fix this in this PR.

[lxning]

@agunapal reply to question:

let's add sth like this at https://github.com/pytorch/serve/blob/master/frontend/server/src/main/java/org/pytorch/serve/util/ApiUtils.java#L283:
if (v == 507) {
String msg = "outofmemory"
} else {
}

[agunapal]

@lxning Looks like that wasn't the right place. v value was 413 there. We can perhaps address this in another PR in interest of time?