Skip to content

Commit f151a3d

Browse files
mschmickingclaude
andcommitted
docs: slim the README and move detail into docs/
The README had grown to 428 lines — good documentation, but too much to read before deciding whether to try the tool, and it is also the npm landing page. It is now 219 lines covering what a newcomer needs: what the problem is, quick start, whether to use this or the VS Code extension, the command table, the safety model, and enough authentication to get a first run working. Everything deeper moved out: - docs/CONFIGURATION.md — config fields, id-to-path mapping, scriptRoot rules, patterns, editor/types setup - docs/AUTHENTICATION.md — password sources, HTTPS, self-signed certs, CI - docs/TROUBLESHOOTING.md — the things that look like bugs and are not, starting with 'logs shows nothing' - docs/DEVELOPMENT.md — running from a clone, tests, invariants, contributing - docs/AUTOMATION.md — expanded with the record shapes, which were previously only discoverable by reading the source The 'Scripting and automation' section was the specific complaint: a long teaser linking to a doc that said little more. The teaser is now four lines and the detail is real. docs/images/README.md is deleted — it described how to take the screenshot, which now exists. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent fa3c83a commit f151a3d

7 files changed

Lines changed: 438 additions & 338 deletions

File tree

README.md

Lines changed: 65 additions & 273 deletions
Large diffs are not rendered by default.

docs/AUTHENTICATION.md

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
# Authentication
2+
3+
How `iob-sync` gets a password, and why it refuses to take one on the command line. See
4+
the [README](../README.md) for the overview.
5+
6+
## Instances without authentication
7+
8+
Nothing to do. `iob-sync` probes `<url>/login`; a 404 means authentication is disabled
9+
and no credential is needed.
10+
11+
## Instances with authentication
12+
13+
ioBroker will not accept a password over plain HTTP, so an authenticated instance is
14+
also on HTTPS — usually with a self-signed certificate. Set `allowSelfSigned: true`
15+
(interactive `init` offers this automatically), then save a password once:
16+
17+
```bash
18+
iob-sync login # prompts without echoing, verifies, then saves
19+
```
20+
21+
`login` checks the password against the live instance **before** storing it, so a typo
22+
fails immediately rather than on the next command. `iob-sync logout` removes it.
23+
24+
## Where the password is kept
25+
26+
**Never in the project.** `.iobroker-sync.json` holds only the username. The password
27+
goes to `~/.config/iobroker-sync/credentials.json`, mode `0600` inside a `0700`
28+
directory, keyed by URL and username so several instances can be used from one machine.
29+
30+
Override the location with `IOBROKER_SYNC_CREDENTIALS` — the test suite does this so it
31+
never touches a real store.
32+
33+
## There is no `--password` flag
34+
35+
Deliberately. `argv` is readable by any other local process via `ps`, and shells record
36+
it in history. The sources actually supported, in the order they are tried:
37+
38+
| Source | Use |
39+
| ------------------- | ---------------------------------------------------------------------- |
40+
| `--password-stdin` | scripts and CI: `printf '%s' "$PW" \| iob-sync --password-stdin login` |
41+
| `IOBROKER_PASSWORD` | ad-hoc shells |
42+
| saved credentials | normal interactive use, after `iob-sync login` |
43+
| hidden prompt | when nothing else is available and a terminal is attached |
44+
45+
Prompts require stdin **and** stdout to be TTYs, so a script, CI job or agent gets a
46+
clear error instead of hanging on an invisible prompt.
47+
48+
## Which login path is used
49+
50+
OAuth2 (`POST /oauth/token`) is tried first, falling back to the legacy `POST /login`
51+
session cookie used by older Admin versions. `--verbose` reports which one worked and
52+
where the password came from:
53+
54+
```
55+
debug: authenticated via OAuth2 (/oauth/token), password from store
56+
```
57+
58+
The legacy path is covered by tests but has never run against real hardware — current
59+
Admin authenticates via OAuth2.
60+
61+
## Ports and certificates
62+
63+
Connect to the **admin adapter** port, usually 8081 — not the socket.io adapter port
64+
(8084), which lacks the permissions this tool needs.
65+
66+
`allowSelfSigned` applies to both the HTTPS login and the websocket. It accepts any
67+
certificate; there is no way to pin a specific one. Without it, an untrusted certificate
68+
fails before any credential is sent, and the error names the setting that fixes it.

docs/AUTOMATION.md

Lines changed: 47 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,13 @@
11
# Scripting and automation
22

3-
Machine-readable output, and what it is for. See the
4-
[README](../README.md) for everything else.
3+
Machine-readable output, and what it is for. See the [README](../README.md) for the
4+
overview.
55

6-
Human output is for reading; `--json` is for everything else. It emits
7-
[NDJSON](https://ndjson.org) on stdout — one JSON object per line, each tagged with a
8-
`type`. Human text is suppressed and warnings/errors stay on stderr, so **stdout stays
9-
parseable even when a command fails**.
6+
`--json` puts [NDJSON](https://ndjson.org) on stdout — one JSON object per line, each
7+
tagged with a `type`. Human text is suppressed and warnings/errors stay on stderr, so
8+
**stdout stays parseable even when a command fails**.
109

11-
What that is actually for:
10+
## What it is actually for
1211

1312
```bash
1413
# Fail a CI job if anything drifted from git
@@ -25,10 +24,45 @@ iob-sync --json logs --level error
2524
```
2625

2726
It is also what makes the tool usable by a coding agent: an agent editing your scripts
28-
can read `status`, push, and then watch `logs` for a compile failure, without
29-
screen-scraping a table meant for a terminal.
27+
can read `status`, push, then watch `logs` for a compile failure, without screen-scraping
28+
a table meant for a terminal.
3029

31-
Records carry underlying values rather than display strings — `enabled` is a boolean, not
32-
`"✓"`; `engine` is the full instance id, not `js.2`; and `--json status` includes the
33-
in-sync scripts the human view collapses into a count. NDJSON rather than one array
34-
because `logs` and `watch` never end; use `jq -s .` if you want a single document.
30+
## Record shapes
31+
32+
Every record carries a `type` discriminator. Records hold underlying values rather than
33+
display strings — `enabled` is a boolean, not `"✓"`; `engine` is the full instance id,
34+
not `js.2`.
35+
36+
| Command | `type` | Fields |
37+
| -------- | -------- | ------------------------------------------------ |
38+
| `list` | `script` | `id`, `path`, `engine`, `engineType`, `enabled` |
39+
| `status` | `status` | `id`, `path`, `state` |
40+
| `pull` | `pull` | `id`, `path`, `dryRun` |
41+
| `push` | `push` | `id`, `path`, `created`, `dryRun` |
42+
| `diff` | `diff` | `id`, `path`, `state`, `against` (snapshot mode) |
43+
| `backup` | `backup` | `snapshot`, `createdAt`, `scripts`, `entries` |
44+
| `logs` | `log` | `message`, `severity`, `from`, `ts` |
45+
46+
`--json status` includes the in-sync scripts that the human view collapses into a bare
47+
count, so a consumer never has to re-run with `--verbose`.
48+
49+
`dryRun` is present on anything that would have written, so a consumer cannot mistake a
50+
`--dry-run` rehearsal for a real change.
51+
52+
## Why NDJSON rather than one array
53+
54+
`logs` and `watch` never end. An array could never be closed, and nothing would appear
55+
until the process exited. Line-at-a-time also means an agent watching stdout sees
56+
progress as it happens.
57+
58+
Use `jq -s .` to slurp the stream into a single document when you want one:
59+
60+
```bash
61+
iob-sync --json list | jq -s 'map(select(.enabled)) | length'
62+
```
63+
64+
## Exit codes
65+
66+
`0` on success, `1` on a user-facing failure — a conflict needing `--force`, a missing
67+
config, a failed login. Errors go to stderr as human text, never as NDJSON, so a non-zero
68+
exit with empty stdout is unambiguous.

docs/CONFIGURATION.md

Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
# Configuration
2+
3+
Where files live, how ioBroker ids map to paths, and how to give your editor
4+
intellisense. See the [README](../README.md) for the overview.
5+
6+
## The project folder
7+
8+
`iob-sync` is a global command that works on a **project folder** — the folder holding
9+
`.iobroker-sync.json`. Where the tool itself is installed is irrelevant; nothing is ever
10+
read from or written to `node_modules`.
11+
12+
```
13+
~/iobroker-scripts/ <- project root: run iob-sync anywhere inside it
14+
├── .iobroker-sync.json <- config. Commit it; it holds no password.
15+
├── .iobroker-sync/ <- state, backups, trash. Gitignored; may contain secrets.
16+
└── scripts/ <- scriptRoot: your scripts land here
17+
├── common/garage.ts
18+
└── Switch-Musiccast.js
19+
```
20+
21+
Commands search upward from the current directory for `.iobroker-sync.json`, the way
22+
`git` finds `.git`, so you can run them from any subfolder. `-C <dir>` runs as if started
23+
somewhere else.
24+
25+
## `.iobroker-sync.json`
26+
27+
| Field | Meaning |
28+
| ----------------- | ------------------------------------------------------------------ |
29+
| `url` | Admin base URL including scheme and port, e.g. `https://host:8081` |
30+
| `scriptRoot` | Folder for synced scripts, relative to the project root |
31+
| `allowSelfSigned` | Accept an untrusted TLS certificate |
32+
| `username` | Admin username, or `null` when authentication is disabled |
33+
| `defaultInstance` | javascript instance assigned to newly created scripts |
34+
35+
Commit this file — it holds no password. Credentials live outside the project entirely;
36+
see [AUTHENTICATION.md](AUTHENTICATION.md).
37+
38+
## `scriptRoot` cannot escape the project
39+
40+
Absolute paths and `../` are rejected. `scriptRoot` is the directory the tool writes
41+
into, so a config pointing outside could drop files anywhere on your disk.
42+
43+
This means you do not point an existing project at a scripts folder elsewhere — you run
44+
`init` **in** the folder you want to keep scripts in:
45+
46+
```bash
47+
cd ~/iobroker-scripts # your git repo
48+
iob-sync init # config lands here, scriptRoot defaults to "scripts"
49+
```
50+
51+
### If that folder already contains files
52+
53+
Safe, but worth knowing: when a script would land on top of a file you already have and
54+
the two differ, `pull` reports a **conflict** and leaves your file alone. `--force` takes
55+
the server's copy. Identical files are adopted silently.
56+
57+
Setting `scriptRoot` to `.` works, but scripts then land beside your `README.md` and
58+
`package.json`. A subfolder is tidier and keeps `status` output readable.
59+
60+
## How ids map to files
61+
62+
| ioBroker object | local file |
63+
| ---------------------------------------------- | ----------------------------- |
64+
| `script.js.common.garage` (`TypeScript/ts`) | `scripts/common/garage.ts` |
65+
| `script.js.Switch-Musiccast` (`Javascript/js`) | `scripts/Switch-Musiccast.js` |
66+
67+
Script folders are ioBroker `channel` objects; nested folders map to nested directories.
68+
`Blockly` and `Rules` scripts are pulled as `.block` / `.rules` for completeness, but
69+
their sources are generated XML/JSON and are not meant to be hand-edited.
70+
71+
Sources are normalised to LF before hashing and upload, so a CRLF checkout does not show
72+
every script as modified.
73+
74+
## Patterns
75+
76+
Patterns match case-insensitively against both the ioBroker id and the local path. A
77+
pattern without `*` matches as a substring (`iob-sync diff garage`); one containing `*`
78+
is an anchored glob (`iob-sync status 'common/*.ts'`).
79+
80+
## Editor support
81+
82+
Straight after a `pull` an editor does not know what `log`, `schedule`, `on` or
83+
`getState` are — they exist only inside the javascript adapter's sandbox, so every script
84+
shows `Cannot find name 'log'`. Fix it once:
85+
86+
```bash
87+
iob-sync types
88+
```
89+
90+
That downloads the adapter's own typings to `.iobroker/types/` and writes
91+
`<scriptRoot>/tsconfig.json`, so any LSP client — neovim, VS Code, Helix, Zed — picks
92+
them up. **Restart your language server afterwards.**
93+
94+
`init --types` does the same during setup. `iob-sync types` exists so you can add or
95+
refresh them later without touching a working config. `--force` replaces an existing
96+
`tsconfig.json`; `--offline` skips the download.
97+
98+
### Why the generated config sets `moduleDetection: force`
99+
100+
Each ioBroker script runs in its own sandbox scope, but to TypeScript a folder of plain
101+
scripts shares one global scope — so two scripts each declaring `const helper` would
102+
collide with TS2451, an error about code that is perfectly fine at runtime. Module
103+
semantics give every file its own scope, matching how the adapter actually runs them.
104+
`module`/`target` are `es2022` rather than `commonjs` because the adapter permits
105+
top-level `await`, which commonjs rejects with TS1378.

docs/DEVELOPMENT.md

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
# Development
2+
3+
Running `iob-sync` from a clone, and working on the tool itself. See the
4+
[README](../README.md) for what it does.
5+
6+
## Running it from a clone
7+
8+
Until the package is on npm, this is how to try it:
9+
10+
```bash
11+
git clone https://github.com/mschmicking/iobroker-sync
12+
cd iobroker-sync
13+
npm install
14+
npm run build
15+
npm link # puts `iob-sync` on your PATH from your working copy
16+
```
17+
18+
**Then leave this directory.** `iob-sync` operates on whatever folder you run it in, so
19+
`init` belongs in your _scripts_ folder, not in the clone:
20+
21+
```bash
22+
cd ~/iobroker-scripts # your own folder — create it if it does not exist
23+
iob-sync init
24+
iob-sync pull
25+
```
26+
27+
Running `init` inside the clone makes the tool's own repository the project root, and
28+
`scriptRoot` cannot point outside it — so there is no way to reach scripts kept
29+
elsewhere. If you have already done that, delete the stray `.iobroker-sync.json`,
30+
`.iobroker-sync/` and `scripts/` from the clone and start again in the right folder.
31+
32+
If you would rather not `npm link`, call the built entry point directly and let `-C`
33+
choose the project:
34+
35+
```bash
36+
node /path/to/iobroker-sync/dist/cli.js -C ~/iobroker-scripts list
37+
```
38+
39+
## Working on the tool
40+
41+
```bash
42+
npm test # build + full suite
43+
npm run test:unit # pure-logic tests only
44+
npm run lint
45+
npm run format
46+
npm run verify # lint + format check + typecheck + tests
47+
```
48+
49+
`npm test` builds `dist/` as well as `dist-test/`, because `test/cli.test.ts` spawns the
50+
real binary.
51+
52+
Every test runs against an in-process fake Admin server (`test/fake-server.ts`); **no
53+
test may touch a real instance.** The TLS suite generates a self-signed certificate on
54+
first run via `openssl` and caches it under `test/fixtures/`; without `openssl` it skips
55+
rather than fails.
56+
57+
## Invariants the tests exist to protect
58+
59+
Do not weaken these without a deliberate decision:
60+
61+
- `pull` never deletes a local file, and never silently overwrites one.
62+
- `push` sends only `common.source` and `common.engineType` — never `enabled` or
63+
`engine`. `ObjectsApi.extendScript` is typed to enforce it, so a sync bug structurally
64+
cannot stop a running script or move it between javascript instances.
65+
- Conflicts refuse and exit non-zero rather than guessing.
66+
- Only `remove`, `rename` and `move` delete anything; each needs `--yes` and writes the
67+
object to `.iobroker-sync/trash/` first.
68+
- `watch` suppresses the adapter's own `compiled`/`sourceHash` write-back. A regression
69+
there means an infinite push loop against a live instance.
70+
71+
Commands never call `console.*` — output goes through `ctx.log`, which is what makes
72+
`--json` and the tests possible. ESLint enforces this everywhere except `cli.ts`.
73+
74+
## Contributing
75+
76+
Pull request titles must be [conventional commits](https://www.conventionalcommits.org/)
77+
(`feat:`, `fix:`, `docs:`, `chore:` …) — the title becomes the squashed commit message
78+
and drives the version bump, and a workflow validates it. PRs are squash-merged.
79+
80+
**[AGENTS.md](../AGENTS.md)** has the full picture: the Admin wire protocol, why each
81+
lint rule is on or off, and what is deliberately untested. Read it before changing sync
82+
logic.

0 commit comments

Comments
 (0)