Architecture: One machine per User with Apps

[mguidon]

Give every user his own machine with his favorite services per-installed as apps.

[sanderegg]

In a billable service the following currently happens:

Current situation billable:

• using the pricing plan, the machine type is added as a placement constraint on the dynamic service
• when starting the service the service resources are overridden with the available resources of the machine type (e.g. g4dn has 32 CPUs, 32 RAM --> it gets set on the service reservations/limits)
• every service gets its own machine with no sharing

Current situation non-billable:

• when starting a service the service resources are used as defined as reservations/limits
• any number of service may run on a machine, thus sharing the machine resources
• the services are randomly placed on the machine thanks to the docker swarm orchestrator

What we would like to have:

• we want to be billable,
• we want services of the same user/group/wallet/project to run on the same machine(s) (e.g. no sharing of the machine with another user)

What need to change for this to work:

• in a billable model we should NOT override with the machine resources, but keep the service assigned resources?
• make sure that the user's services are all running on the same machine (that means placement constraint, based on user, group, wallet, project or a combination of them all?)
• do we want to specify machine type as well? (then all the selected services should have the same type in order to work)
• the autoscaling app must recognize these additional placement constraints and set them as node labels on the machine,
• potentially removing them when it detects that no single service like that is still up,
• that also means that if only 1 service is still running it will be running on a rather expensive machine.
• Of course the current system a la sim4life.io shall continue to work seamlessly

[mguidon]

• Can we set the reservation so low that all services would run but set the limit to the max given by the ec2 machine type?
• Selecting machine type: Maybe we can parse the resrource specs and see if we need VRAM
• First use case would probably be:
  • Create a template with a pipeline and set it up such that it can run on one node
• Later on: Let the user multiselect some services in the GUI?

[sanderegg]

• yes we can set that so, but then depending on what you do in 1 of the service you can blow up the other services (say you use all the RAM in service 1, then what happens to the other services? I think we already went through that pain several times and decided against having too big a difference between reservations and limits)
• yes that is doable, currently the autoscaling only checks the CPU/RAM but it could also check the VRAM need, but then how do you choose between for example g4dn and g5?
• and what I forgot to mention is that it needs to still work for sim4life.io style... will add it in the list
• what should happen when 1 service needs g4dn, the other g5, the other t3, and the last one yet another?

[sanderegg]

option 1 would be:

• somehow define in a template UI what machine shall be used in the project
• then it's the user responsibility to have a machine big enough for all the services he/she wants to run there (1 at a time still)
• it is the user responsibility to use the correct resources as they will not be overwritten on start (vs sim4life.io that does it)
• the services would start with 0 reservation and their defined limits
• startup must remain backward compatible

Foreseen issues:

• if multiple services run in parallel they might be killed by the host if for instance 2 of them are at 60Gb and the host only has 100Gb

[sanderegg]

issue linked to this discussion:
#5669