Additonal Considerations¶
PubApps Virtual vs Shared Hosting Infrastructure¶
As you can see from the architectural diagram the approach we took to the PubApps infrastructure is a 'curated cloud like experience' for web application deployments. We provide a platform, but it's up to you to choose how to deploy and run your application to make sure you work is productive and the application is stable, scalable, and efficient. You should consider what resources to use for the application front-end, back-end, and any worker type processes and select carefully from the options that include
Scheduled Resources¶
The application can submit SLURM jobs to the SLURM setup in PubApps or the HiPerGator SLURM scheduler in PrivApps, so you can develop and use the same code for both sides or even develop the app on HiPerGator before deploying into PubApps. * Pro: The advantage is a stable interface using standard slurm commands like sbatch, squeue, and scancel or the industry-standard DRMAA interface like python-drmaa, which we use ourselves and encourage you to check out. * Con: The disadvantage is that there can be contention for the resources to run the jobs, so you have to implement an asynchronous approach to running workfloads and be able to notify the users when the work is done to deliver the results.
Shared Resources¶
The pubcontainer nodes are large servers that can host many web applications whether deployed via simpler methods like conda environments or with full Podman orchestration (Docker is not available as it conflicts with RHEL-native podman in the system images). These nodes also have attached GPUs that are assigned to particular pubapps instances when purchased. * Pro: The advantages include being able to run long-term workloads from the web application backend processes without developing additional methods to submit scheduled jobs. As pubcontainer nodes are shared systems even if an application exceeds the resources that were paid for in short bursts there is enough capacity in the servers that the performance of other applications should not be affected. * Con: There is a limited number of GPUs available for purchase and use by the web applications. There is strict control over GPU access. There is a potential of excessive resource use to affect other applications.
VM Resources¶
Using a standalone VM (Virtual Machine) is a good solution for when the project needs dedicated resources, has more stringent security requirements, and may be running applications that are created to serve their every component (web application itself, database server, other services) from a single machine, etc. In short, if an application doesn't play well with others it should be in its own VM. On the PrivApps side, any application that has requirements beyond being a lightweight front-end interface that does database querying or is only used to submit jobs to SLURM and display the results (e.g. CryoSPARC) has to be hosted in a VM. * Pro: The advantages include stronger isolation from other applications, both security and resource-wise, defined resources. * Con: The disadvantages include longer setup time as each VM has to be provisioned by the DevSecOps group, resource constraints as there are no shared resource to burst to, no GPUs.
Software environment¶
All PubApps and PrivApps VMs and servers run RedHat Enterprise Linux 8 Operating System. You can use conda environments and containers (Podman and Apptainer/Singularity) for the applicatio deployment.
CI¶
We suggest setting up CI (continuous integration), so code changes in an external git repository get applied to the application. This approach allows limiting access to the hosting environment while a potentially large number of developers can work on the application code locally.