Integration of website/documentation/sample/code GitHub repos

[twallema]

Food for thought

1. Why does the FlepiMop project consist of four GitHub repositories instead of just one with an appropriate subdirectory structure?

   • FlepiMoP: source code
   • flepimop-documentation: gitbook documentation
   • flepimop_sample: a tutorial/minimum working example
   • flepiMoP_site: website, seems to act as wrapper for gitbook (flepiMoP_site/_data/navigation.yml)

   I was taught to use just one GitHub with the following suggested structure (see https://github.com/yngvem/python-project-structure/tree/master; or see my code twallema/pySODM) to accomplish the same task.

      flepiMoP
      ├── docs
      │   ├── source (flepimop-documentation)
      │   └── website (flepiMoP_site)
      ├── tutorials (flepimop_sample)
      │   ├── minimal_working example
      ├── src (flepiMoP)
      │   └── flepiMoP
      │       └── __init__.py
      ├── tests
      │   └── __init__.py
      ├── .gitignore
      ├── LICENSE.txt
      ├── README.md 
      ├── setup.py
      └── flepimop-env.yml

2. There are two websites

   For a newbie it is confusing there are two websites: https://www.flepimop.org/ + https://iddynamics.gitbook.io/flepimop. Based on flepiMoP_site/_data/navigation.yml, the first website seems to wrap the second under the Documentation tab.

[saraloo]

Quick FYI: The documentation and sample repo have been merged in to the flepiMoP repo as of last week.

[jcblemai]

Thanks for raising the issue. The multi-repo approach had some historical sense in the way this project was managed and what as public or not. Most of these requirements were removed.

Last week, we merged the sample_repo and the documentation within this main repository:

• Documentation gitbook #225
• add the flepiMoP sample repository and run integrations against it #227

and we added an integration test for the examples. These changes still need to be reflected in the documentation (leaving this issue open until it's done).
There are still some open issues while consolidating: we want to keep the repository very light and without git-lfs. Our former version, COVIDScenarioPipeline, was several Gbs and cloning took several minutes, which is annoying as some batch jobs clone dozens of times on provisioned AWS instances. So what do we do about documentation assets and pictures ? and heavy notebooks? How do we teach folks how to use the USA repository if the sample repo is within flepiMoP (this is solved by having command line entry points).

The website being in a different repo makes sense to me, and very rarely do open-source projects have their website in the main repository (see e.g pytorch has https://github.com/pytorch/pytorch.github.io) especially when it's a github.io website.

Regarding the two websites, I agree that it would be ideal to have the documentation hosted in our own domain, but there are good arguments to let it be hosted on gitbook (search, non-static content, ease of development). Note that other packages (e.g https://xarray.dev) also open the documentation in another tab but in that case, it's a subdomain which is much nicer. Perhaps we could do something like this with gitbook. @clifmckee @shauntruelove who owns the flepimop.org domain ? We could have the documentation lie at docs.flepimop.org if you can add a specific CNAME entry to it (see https://docs.gitbook.com/publishing/custom-domain/configure-dns, happy to help).

Regarding the project structure, flepiMoP python's package gempyor follows it (https://github.com/HopkinsIDD/flepiMoP/tree/main/flepimop/gempyor_pkg) but right now the project is still a mix of R and python with the entry point written in R (that will hopefully change). The yngvem repo is 5y.o and the newer PEP guidelines for packaging are here: https://packaging.python.org/en/latest/discussions/src-layout-vs-flat-layout/ (linking specifically to this page on whether to name our package directory src or flepiMoP.