Setting up dev environment on Ubuntu

[quettabit]

There seem to be a list of things that needs to be worked on to make devenv-start work
seamlessly on Ubuntu machines with different environment managers.

1. apt-get install docker actually installs a docking application (system tray). However apt-get install docker-compose installs the docker that we need because of its dependency.
2. So, the right ubuntu package name for docker is docker.io. There is another concern over here which needs to be figured out. docker.io is released and managed by Debian/Ubuntu and is not from Docker. docker-ce is the one from them. Learn more about docker.io vs docker-ce. So the rate at which docker.io gets updated is different from docker-ce.
   • When apt-get install docker.io docker-compose is done. Below are the outputs for two commands.
     • docker -v gives Docker version 19.03.6, build 369ce74a3c.
     • docker-compose -v gives docker-compose version 1.17.1, build unknown
   • Similarly for apt-get install docker-ce docker-compose,
     • docker -v gives Docker version 19.03.11, build 42e35e61f3.
     • docker-compose -v gives docker-compose version 1.17.1, build unknown
3. Currently, docker-compose.yaml has its version as 3.7. However, it doesn't seem to be supported by the version installed by apt-get. So I had to install docker-compose like how it is been said in their website. Now,
   • docker-compose -v gives docker-compose version 1.26.0, build d4451659

Now, all is fine with respect to launching docker containers. Moving on to issues with respect to Python environment mangers especially conda. We need to fix them for atlas to work as expected.

1. The line python startup_atlas_api.py ${ATLAS_PORT} in start-devenv.sh usually picks up the system set python binary (which doesn't have to be 3.7). However, we would need it to pick the python binary installed within the foundations conda environment. A quick fix that worked for me is to add a line to source activate foundations environment which has desired python version.

2. source activate_dev_env.sh sets PYTHONPATH. However, it doesn't get picked up by the python process that gets started for startup_atlas_api. A quick hack was to set PYTHONPATH from start-devenv.sh before executing startup_atlas_api.

Now, the services running as expected. However, there seem to be a bug when submitting a job to scheduler. Not sure if it is transient.

1. atlas/foundations_local_docker_scheduler_plugin/src/foundations_local_docker_scheduler_plugin/bundle_deployment.py and in few other files, the key for scheduler_url in config is "scheduler_url". But job submission worked only after changing few of them to "SCHEDULER_URL".

Now, atlas works as expected i.e., job submission is successful and project can be seen on UI.

With all being said, I would like you guys to comment on how to proceed further to make the dev environment setup process smooth on Ubuntu machines, and to update the documentation accordingly. I can work on it once we reach a consensus.