Computer use
Computer use is a Labs feature. It is built into nightly builds and switched off until you turn it on in Settings → Labs.
Some things have no API. The setting is four clicks into a console, the report only exists behind a login, the phone app has no other way in. Computer use lets Celeris work those surfaces the way you would: look at what is on screen, click, type, scroll, look again.
It is the slowest and least reliable way for Celeris to do anything, and it is the only way to do some things. Reach for a tool or connector first, and for this when there is not one.
Targets
Pick what Celeris may drive from the target picker in the composer.
- A browser Celeris runs itself. An isolated browser window, separate from your own, with a separate profile, separate cookies, and no access to sessions you are signed in to elsewhere. This is the safest target and the right default.
- An Android device. A phone or tablet plugged in with USB debugging turned on, chosen explicitly by device.
- A window on your desktop. One foreground window, on a machine you are sitting at, watching it happen.
Whatever you pick, a step that cannot be undone or leaves your machine still stops and asks. The target chooses what Celeris drives. It does not widen what Celeris may do.
The session
Computer use only happens inside a short, explicit session.
- It is bound to one target and one approval. Approving Celeris to drive a browser does not approve it to drive your phone.
- It has a visible Stop, always. Pressing escape twice quickly does the same.
- It expires when the turn ends. It does not linger, and the next turn starts from nothing.
- It pauses if the target changes underneath it, when a different window comes forward or the device disconnects, rather than carrying on against whatever is now in front of it.
- It can never become an "always allow" rule. Driving your computer is approved one session at a time, by design.
While a session is live, a strip above the composer names the target, what the session is currently allowed to do, and its status.
How Celeris sees the screen
Mostly, not as pixels. Celeris asks the target to describe itself, listing what elements are there, what they are called and what can be clicked, and works from that description. That is faster, cheaper and far more accurate than reading a picture. It takes an actual screenshot when the description is missing or incomplete, or when it needs to zoom in on something visual.
Anything Celeris reads off a screen is treated as data, never as instructions. Text in a web page telling Celeris to do something is not a request Celeris will honour.
When you should not use it
- On a screen showing credentials, payment details or personal records. Whatever is on the target is what Celeris reads; sign in yourself first, off-session, and hand it the page afterwards.
- When you are not watching. This is a watch-it-happen feature.
- As a substitute for a connector that exists. Clicking through an interface is fragile in a way an API call is not; it breaks when a button moves.