Yes, after a user starts an app on their device, the app is available offline.
The created apps run natively, not as a web app on users' devices. Therefore no permanent internet connection is necessary. This is a good scenario for Open as App especially for calculators that need to be served in areas with the poor network coverage.
Make sure you have Open as App installed, you are logged in, and the desired app has been launched at least once on the device before you "go offline".