195 Commits
Author SHA1 Message Date
redtea 916a49958b release: 0.2.4 2026-10-18 12:13:47 +03:00
redtea dde85031aa feat: the text of a picture's or a search's card folds and unfolds smoothly 2026-10-18 12:13:29 +03:00
redtea dd1244a169 feat: pictures for the assistant - the models make and edit them by themselves through the Vertex API (a new picture, or the user's attached photo, or the last picture made), with an API key in Настройки and a switch for the prompt improver; folded descriptions of pictures and searches; the name of an attached file opens or saves it; a deleted chat is wiped from the database files, attachments included 2026-10-18 12:12:54 +03:00
redtea c72fcfebf3 fix: http-lp forwarding - a download the agent finished reading reaches the browser whole; the session closing on its end of stream cut off what was still queued (up to 32 MB), so browsers restarted big downloads 2026-10-18 11:38:05 +03:00
redtea 054fc97da8 fix: the assistant's web search and page reading on by default, as the chat - they refused («ассистент выключен в профиле») anyone who never touched the switch 2026-10-18 11:26:20 +03:00
redtea 114f8b1e1b release: 0.2.3 rebuilt with the Порты fix 2026-10-18 11:22:19 +03:00
redtea a1604206ee fix: Порты - the «Без ufw» switch in the line of the note, at its right, never over it when the button wraps 2026-10-18 11:22:01 +03:00
redtea a107745355 release: 0.2.3 2026-10-18 11:16:25 +03:00
redtea dcbc602d88 feat: Настройки › ИИ-ассистент - each model and the web search folded to a line (name, kind and model), opened when wanted; a new model opens, its line follows its fields 2026-10-18 11:16:08 +03:00
redtea fcf032042b release: 0.2.2 2026-10-18 11:09:16 +03:00
redtea 01f02360c4 chore: the GitLab namespace is itzredtea now 2026-10-18 11:03:31 +03:00
redtea 57de7bf967 chore: move to GitLab - server-install.sh takes the latest release and dist/ from gitlab.com/itzRedTea/lpra, README links and screenshots kept in the repo; a guide for moving other projects 2026-10-18 10:44:18 +03:00
redtea 173f46a145 fix: Qwen Omni gets its files right - laid out by the app's own inputs (audio before image there, they were swapped), a video as Gradio 5 takes one ({video, subtitles}); the model is told it has the files, so it answers about them 2026-10-17 15:42:37 +03:00
redtea 03f963340e feat: «Думать» uses a model's own thinking when it has it (OpenAI-compatible reasoning, Anthropic extended thinking) instead of asking it to think in words too; learned per model, the word-thinking prompt kept for models without; without «Думать» such a model is asked not to think 2026-10-17 15:36:35 +03:00
redtea 37ef886503 feat: the model's short name up to 12 characters 2026-10-17 15:33:19 +03:00
redtea 6c1da79b4a feat: the model's short name up to 9 characters, the chat 408px wide so nine of the widest letters fit 2026-10-17 15:07:49 +03:00
redtea 6a17f40042 feat: a short name of up to 7 characters for each model, on the chat's model button, always whole; the message field back to the normal 14px 2026-10-17 14:58:04 +03:00
redtea 351e4ad28d fix: the chat's message field lower - 42px for a line instead of 52, the clip and send to scale, still centred on the last line 2026-10-17 14:47:42 +03:00
redtea 983809714b feat: several models for the assistant, each with its own name, kind, address and key; a model menu in the chat when there are several (or always, from Аккаунт › Вид), the choice kept for the user 2026-10-17 14:41:34 +03:00
redtea ded86e1c3f fix: the panel's input dialog - a dark field as everywhere, with the accent outline when focused 2026-10-16 22:08:30 +03:00
redtea b4bcc05162 docs: a plan for flow control in the direct tunnel, for when one slow connection stalling an agent's others starts to matter 2026-10-16 22:02:59 +03:00
redtea e8d45f2f7b docs: http-lp forwarding goes over the agent's WebSocket, by long polls only without one - said so in the panel, the code and the README 2026-10-16 21:56:24 +03:00
redtea e93a2bbd3a fix: the stop button blue again, as the plane 2026-10-16 21:51:03 +03:00
redtea e0b3999d0c feat: a chat of the assistant is deleted at once, without asking 2026-10-16 21:47:50 +03:00
redtea 77322e8644 chore: update build 2026-10-16 21:44:43 +03:00
redtea 6085b5a094 fix: the stop square fades away as a red square and becomes the plane only once hidden; the square a little bigger 2026-10-16 21:44:09 +03:00
redtea e0cc5a06b7 fix: server-install.sh - an answer «n» about nginx no longer counts as nginx set up, so without nginx the server listens on every address 2026-10-16 21:24:27 +03:00
redtea 667c6177f4 feat: server-install.sh - install or update the server in one command: the latest release's binary, dist/ from main, the systemd service, nginx with a domain and certbot if wanted, a health check 2026-10-16 20:47:26 +03:00
redtea 86f7c96807 feat: rewrite nginx files 2026-10-16 20:18:06 +03:00
redtea cf0b2baf75 fix: gitignore 2026-10-16 20:15:47 +03:00
redtea 2ab2aa1e38 fix: the message field grows only when the word really wraps (measured without a scrollbar taking its width); the stop button red 2026-10-16 20:04:38 +03:00
redtea 087eabf3b4 feat: «Здесь» from a machine's terminal opens the file pane in the terminal's folder (asked of the agent); Аккаунт › Вид can turn «Здесь» back to the home folder. Version 0.2.1 2026-10-16 18:28:15 +03:00
redtea 57c2f1338f feat: «Здесь» from a machine's file pane opens the terminal in that pane's folder (the agent starts the shell there; an older agent, or a folder gone, at home) 2026-10-16 18:17:11 +03:00
redtea 4fb26a9652 feat: the chat's arrow down midway between the answers' right edge and the chat's border, fading in and out 2026-10-16 18:11:21 +03:00
redtea 46f7f743b3 fix: the clip and the send button keep to the last line of the message, as in Telegram 2026-10-16 18:04:14 +03:00
redtea 70729ef63f feat: folders and files the assistant cannot take, dropped from a file pane, go into the message as their paths; a file that fails shows a red note over the field, fading in and out like an error toast 2026-10-16 18:02:43 +03:00
redtea 2d6b3916c8 fix: a clearer chat mark in the empty chat (two bubbles), a bigger send plane 2026-10-16 17:57:25 +03:00
redtea 35ad2d435b feat: the send button as in Telegram - a blue plane with no plate, coming in only when there is something to send; the empty chat shows a chat mark and «Спросите AI помощника» 2026-10-16 17:56:53 +03:00
redtea 37ae830cbf fix: Safari - drags no longer select page text (user-select needs its -webkit- twin there), the drag ghost follows the pointer without a transition 2026-10-16 17:51:27 +03:00
redtea ffe7c138f1 fix: the send button as far from the right edge as from top and bottom 2026-10-16 17:37:04 +03:00
redtea ae8fffe7d8 feat: fetch for the assistant with Поиск on - a page's main text for the model, the server's own network out of reach; shorter words in Настройки and the chat; the + menu's roots a step left. Also files from panes into the chat, docview and drag fixes (by the user) 2026-10-16 17:32:52 +03:00
redtea 84b42c5369 Add multimodal AI chat attachments 2026-10-16 15:46:23 +03:00
redtea 8aba30a093 fix: the machine's dot in the + menu centred like the other icons 2026-10-16 12:03:11 +03:00
redtea 9337bd26b0 feat: Машины - column names centred over their values 2026-10-16 11:58:33 +03:00
redtea 8bbb2b0ccb fix: machines in the + menu line up with the other items 2026-10-16 11:57:49 +03:00
redtea 80ed7d4ffa feat: a new chat is named by the model in two or three words, asked aside; the machine's name gets its room in the chat's header 2026-10-16 11:23:52 +03:00
redtea 6442c39360 fix: the chat's machine name on one line without the dot, cut with …; close buttons always square 2026-10-16 11:17:32 +03:00
redtea d5d57fbf1b feat: Вид - tab icons слева, справа or отключены; the lists one width 2026-10-16 11:13:12 +03:00
redtea 7ae75e910a fix: «Думал N сек.» holds its arrow inside its highlight, a quiet one 2026-10-16 11:06:31 +03:00
redtea 74a22ab57f fix: the assistant is told which search engine it has, results say which one they came from (and why, when DuckDuckGo stood in), the stand-in is logged 2026-10-16 10:59:39 +03:00
redtea 060975a443 test: a search engine out of its limit gives way to DuckDuckGo 2026-10-16 10:53:53 +03:00
redtea fdd0483286 feat: no note under the command in the assistant's question 2026-10-16 10:48:34 +03:00
redtea fcf9417f5a feat: the assistant's chat - the name clear of the close button, typing dots while the model starts, thinking that opens without reflow; asking before a command: «Выполнить команду на [машина]?», the command in mono, a red «Отмена» 2026-10-16 10:48:05 +03:00
redtea 3641a3627c docs: README - the assistant's search engines 2026-10-16 10:38:22 +03:00
redtea 9912255289 feat: the assistant's search engine in Настройки - DuckDuckGo, a SearXNG of one's own, Brave Search or Tavily, with DuckDuckGo behind the chosen one 2026-10-16 10:38:12 +03:00
redtea 9bf30b281e feat: Поиск - with it on the assistant searches DuckDuckGo itself and answers from what it found; without it, it cannot. «Думал N сек.» 2026-10-16 10:32:33 +03:00
redtea fa62f2c5b6 fix: Думать - the model is reminded in the question itself, the label no longer lands in the arrow, and the thinking with its time is kept to show again later 2026-10-16 10:29:44 +03:00
redtea 3b7a6271ac feat: a dragged tab held over another tab's title opens it to be dropped into; let go with nothing done, the dragged tab comes back 2026-10-16 10:21:37 +03:00
redtea 5216aed237 feat: Аккаунт › Вид - tab icons left, right or hidden, terminal font, compact file lists, the time column, animations; kept per user on the server. Uploads show a ring filling as they go 2026-10-16 10:17:07 +03:00
redtea c003b51b4c feat: a small terminal or folder icon before each tab's name says what it holds 2026-10-16 10:12:00 +03:00
redtea 96163ebf86 fix: dragging files selects no text; without a drag the names select as before 2026-10-16 10:10:08 +03:00
redtea bc1736f105 fix: menus open right by their button again - the page's entrance animation no longer holds a transform that moved fixed menus 2026-10-16 10:09:20 +03:00
redtea b23943ff42 feat: the assistant keeps several chats - the current one named in the header, the list under its arrow to open or delete; actions as compact cards; folding parts open smoothly 2026-10-16 10:06:06 +03:00
redtea 6d8f39f2a5 fix: commands on SSH hosts run under sh -c, so hosts whose login shell is csh or tcsh list, zip and send files too 2026-10-16 10:02:10 +03:00
redtea 458d826de6 fix: the question window as wide as its words 2026-10-16 09:54:13 +03:00
redtea 0c44ef006d fix: a long name in a question is cut with … 2026-10-16 09:39:48 +03:00
redtea 187dc2ae97 feat: PostgreSQL as the panel's database, chosen in Настройки, the SQLite data copied over on its first start; tested on a real PostgreSQL 2026-10-16 09:39:06 +03:00
redtea 055020c3bb feat: links to folders open as folders, on machines and SSH hosts, with their own icon; deleting a link leaves what it leads to 2026-10-16 09:34:32 +03:00
redtea 4da1cc4200 feat: the assistant's answers in Markdown; thinking shows «Думаю…», then «Думал N с», opening smoothly by its arrow 2026-10-16 09:33:23 +03:00
redtea 4b7220fffb feat: account dialog - Выйти beside the name, passkeys last after a line, the add button the field's height; a file pane's status says it is connected 2026-10-16 09:32:04 +03:00
redtea 434900a168 feat: long names in questions cut with … and the window as wide as its words; SSH dialog hides «Вход» for a saved host and says «подключение…» quietly; zip and unzip on SSH hosts without them through Python's zipfile 2026-10-16 09:31:21 +03:00
redtea 6db78399ac feat: the workspace at /workspace; Настройки show the assistant's model as set, key included, and PostgreSQL's fields only when chosen, its two buttons on one line; pages come in softly 2026-10-16 09:30:19 +03:00
redtea 352b016384 fix: the editor shows an empty page, not the last file, while a file loads; text in UTF-16 and with BOM; + menu groups open and shut smoothly; the assistant's errors in plain words 2026-10-16 09:28:00 +03:00
redtea b272aed8fd Fix settings handler build 2026-10-15 22:48:03 +03:00
redtea ce7c417884 feat: the assistant's send is a Telegram-like plane, its height pinned to the field's 2026-10-15 22:38:37 +03:00
redtea bfa38e7d18 feat: the + menu without descriptions, narrower 2026-10-15 22:35:30 +03:00
redtea 00617451aa feat: the assistant's talk is kept in the database per user and the model sees a trimmed window of it; clear is a broom; the ball steps aside for the chat and comes back where the chat was 2026-10-15 22:35:10 +03:00
redtea 92a90646a5 fix: address fields styled like the others 2026-10-15 22:32:36 +03:00
redtea 69600133a4 docs: README - the assistant's actions and modes, saved SSH hosts and their key 2026-10-15 22:30:09 +03:00
redtea 3044e650d3 feat: on a computer the assistant's ball and chat can be dragged, the chat opening beside the ball; on a phone a smaller ball further from the edges; the send button the field's height 2026-10-15 22:29:55 +03:00
redtea ed77c8e0c1 style: gofmt 2026-10-15 22:28:59 +03:00
redtea 7d1996dd7d perf: several dragged items cross to another place as one tar stream; older agents get them one by one 2026-10-15 22:28:46 +03:00
redtea 6916af4236 perf: SSH uploads fetch pieces ahead and downloads post them side by side 2026-10-15 22:25:54 +03:00
redtea 8d7489b577 feat: the assistant's Агент and Думать modes as pills; thinking shows folded above the answer 2026-10-15 22:24:29 +03:00
redtea 847bcd68c0 perf: the browser terminal's WebSocket deflates larger messages 2026-10-15 22:22:59 +03:00
redtea a7e43ab700 feat: assistant reads, lists and writes files and runs commands on any of the user's machines; agent mode does the steps itself, asking before each command and write 2026-10-15 22:21:54 +03:00
redtea d9c7c2dd42 feat: the assistant's chat slides open and shut, smaller on a computer, a screen of its own on a phone that «назад» closes; a sparkle button 2026-10-15 22:19:23 +03:00
redtea 94bf292494 feat: saved SSH hosts per user, the way in sealed with AES-GCM under a server key kept beside the database 2026-10-15 22:18:27 +03:00
redtea 1f930e5aea feat: the panel's own confirm and prompt dialogs everywhere; the assistant's field is one line with a square send button 2026-10-15 22:16:02 +03:00
redtea 3f113ce8d6 feat: on a phone the header's links scroll under a pinned profile button; the assistant's close is red 2026-10-15 22:11:18 +03:00
redtea 46948f5ddc feat: AI assistant (MVP) - chat button per user, sees the workspace, streams answers, runs suggested commands in the active terminal; Gradio, OpenAI and Anthropic models set in Настройки 2026-10-15 22:10:47 +03:00
redtea 9572a35ad5 feat: on a phone Вверх, the path and Обновить share one line; the header is one line without the logo 2026-10-15 22:10:46 +03:00
redtea 64ce8f2b72 fix: ended page scopes start no timers; SSH transfers keep their connection from the idle sweep; linked files download whole; only the sent item unpacks on an SSH host, replace and skip as on a machine 2026-10-15 22:05:04 +03:00
redtea e05ff05015 feat: on a phone the selection's zip and delete sit on the line of the blue buttons; moved to the next line they start on the left 2026-10-15 22:01:17 +03:00
redtea 01dc45ecad feat: file panes follow the machine going offline and back, reconnect a lost SSH host; zip and unzip over SSH where the host has them; phone keys only for terminals; version 0.2.0, README 2026-10-15 21:58:09 +03:00
redtea d92696226e feat: machine pages Порты, Статус, Изменить as dialogs over the current page; Действие column in the ports table 2026-10-15 21:49:18 +03:00
redtea 2009796421 feat: the + joins a lone file pane's top bar, beside Обновить and above the delete button, growing in smoothly 2026-10-15 21:44:15 +03:00
redtea f2740c50b4 feat: drag files between file panes (move or copy, SSH hosts too) and into a terminal of the same machine 2026-10-15 21:42:21 +03:00
redtea a30b89f1d8 feat: square icon buttons for up and refresh in file panes; the Терминал view is named WorkSpace 2026-10-15 21:33:14 +03:00
redtea 7e4daeda4c feat: file manager on SSH hosts through a machine - open from the plus menu 2026-10-15 21:32:18 +03:00
redtea 2859e69abe feat: agent keeps SSH file sessions and runs file operations on the host by commands 2026-10-15 21:26:11 +03:00
redtea f5d6bd30e8 feat: file manager as a workspace pane, side by side with terminals; zip and delete of the selection as icons 2026-10-15 21:24:10 +03:00
redtea 012c0af7f9 feat: workspace lives in the terminal view and keeps its sessions across pages; terminal panes split out 2026-10-15 20:10:28 +03:00
redtea 113c63fd1b feat: one-page panel - shell with Машины, Терминал and Настройки, pages as fragments, forms sent in place 2026-10-15 20:05:20 +03:00
redtea 7b9daa524e feat: workspace helpers - quoting paths, what a dropped file does, with node tests 2026-10-15 20:00:26 +03:00
redtea 0ada0899cf docs: 0.2.0 workspace implementation plan 2026-10-15 19:58:32 +03:00
redtea 8e2028e7f9 docs: workspace design approved, version 0.2.0 2026-10-15 19:52:43 +03:00
redtea 698bc68f99 docs: 0.1.3 workspace design - one-page shell, tabs with terminal and file panes (machines and SSH hosts), drag and drop, machine dialogs 2026-10-15 19:51:02 +03:00
redtea 0fa3de757c feat: edit SQLite databases in the file manager - the agent opens them through SQLite: values by a click, rows added and deleted, an SQL tab; changes go to the audit log 2026-10-15 16:50:56 +03:00
redtea 0279c1d887 fix: corner plus also slides 6 px left over a split tab 2026-10-15 16:42:27 +03:00
redtea 0ccac0bc82 fix: scrollbars darker 2026-10-15 16:38:23 +03:00
redtea e5a45b872a fix: scrollbars more transparent 2026-10-15 16:35:38 +03:00
redtea 9e684605ef fix: terminal scrollbar centred between the plus and the frame 2026-10-15 16:34:39 +03:00
redtea 13897c7478 fix: terminal scrollbar 2 px further left again 2026-10-15 16:34:01 +03:00
redtea 4d2cfe8244 fix: terminal scrollbar 2 px further left 2026-10-15 16:33:06 +03:00
redtea 25ffa60d09 feat: Mac-like scrollbars - thin, no track, shown while scrolling or at the edge and fading out, none when there is nothing to scroll; a lone terminal's bar sits by the frame, right of the plus 2026-10-15 16:32:51 +03:00
redtea baee90b564 fix: terminal corner plus moves left and down only on Windows, clear of its wide scrollbar 2026-10-15 11:47:34 +03:00
redtea 79828e3f8a feat: installer fetched from the direct address sets it as the agent's own with the main one as fallback (and downloads from it); agents fall back to whichever address is not theirs; (direct) beside the connection status 2026-10-15 11:34:58 +03:00
redtea 021da93229 feat: user roles - admins and users limited to the machines given to them, user editor with machines, password sign-in and its lock; direct address without Cloudflare as the agents' fallback and in the installer; version 0.1.2 2026-10-15 10:56:19 +03:00
redtea 36dc12b69d feat: terminal copy and paste by keys off a Mac - Ctrl+C copies a selection (interrupts without one), Ctrl+Shift+C copies and stays from the browser, Ctrl+V and Ctrl+Shift+V paste 2026-10-15 10:45:58 +03:00
redtea 1297ab36d0 fix: transfer dialog keeps its body and shows a spinner and progress while sending; LPRA in the header leads to the machines; no settings link on the settings page; corner plus clear of the scrollbar 2026-10-15 10:44:10 +03:00
redtea fd382d6313 feat: PIN lock - logout left of zero, erase and logout in a muted red 2026-10-15 10:41:22 +03:00
redtea fc488717d3 feat: login - passkey button under the password form, the sign-in button full width 2026-10-15 10:39:00 +03:00
redtea dbb46173a3 feat: dialogs open and close smoothly, the backdrop fading with them 2026-10-15 10:38:02 +03:00
redtea b47e824692 fix: quick reconnect after sleep or a network outage - the tunnel notices a dead connection, retries every 5 s at most, the WebSocket returns as soon as the server answers; version 0.1.1 2026-10-15 00:07:32 +03:00
redtea 5eca9ca215 fix: agent update runs as root when it can (sudo without a password otherwise), checks the download, restarts reliably and reports its result; one-colour file icons; zip and delete on one line on phones; PIN buttons alike; no key fingerprint in the SSH status; corner plus clears split panes 2026-10-14 23:54:24 +03:00
redtea 467bd3cb4f docs: README - SSH terminal, sending to other machines, PIN, passkeys, WebSocket transport, Cloudflare and nginx notes 2026-10-14 21:30:39 +03:00
redtea 18d6684c55 feat: send or move files and folders to another machine or an SSH host from the move dialog, streamed as tar with progress, speed and cancel; same conflict rules as a local move 2026-10-14 21:29:25 +03:00
redtea 1a38c2ac18 feat: terminal tabs a little wider by default 2026-10-14 21:20:37 +03:00
redtea 5a4dc596d5 feat: SSH terminal - the agent connects by SSH to another host (password, key file or its own keys) and the shell opens in a tab with the connection's progress shown; host keys trusted on first use, a changed one refused 2026-10-14 21:20:18 +03:00
redtea 5a204ae9bb feat: a device can skip the PIN for 5, 15, 30 or 60 minutes after it was entered (off by default, this browser only) 2026-10-14 21:13:50 +03:00
redtea 119b524825 feat: optional 4-digit PIN asked in every new tab (not on reload), on-screen keypad for phones; set, change or remove it in the account; five misses sign out 2026-10-14 21:12:09 +03:00
redtea 29353e5e6a feat: install code of 3 to 24 letters and digits with a button to make a new one; guesses are limited per address 2026-10-14 21:07:36 +03:00
redtea 47f1b2e55b feat: the browser terminal on a WebSocket too, long polls when it does not connect or drops 2026-10-14 21:06:32 +03:00
redtea 73bb720ec6 feat: agents connect by WebSocket, falling back to long polling and trying again each minute; auto, ws or lp per machine (0.1.0) 2026-10-14 21:04:13 +03:00
redtea 6dd1120a41 feat: the password is turned off by hand in the account, with a passkey in place; password change; tabs sized by their titles 2026-10-14 20:46:35 +03:00
redtea 67222e2e05 feat: terminal tabs - several machines at once, up to four terminals split in a tab by drag and drop, resizable, renamable; the top bar keeps its status in sight 2026-10-14 20:42:41 +03:00
redtea e7b5d28470 feat: sign in with passkeys, no password once an account has one; an account button and dialog in place of the sign-out button 2026-10-14 20:33:31 +03:00
redtea d2d6fe0100 fix: the copy menu shows as soon as text is selected, a little higher above the extra keys 2026-10-14 20:27:21 +03:00
redtea e983fc179a feat: move or copy dialog with folder suggestions and a rule for what is already there 2026-10-14 20:25:59 +03:00
redtea 89d5bfd3ab feat: copy buttons for the install commands, icons on the file bar buttons; the path title back on the left, the upload figures sliding, dialogs focused without a ring 2026-10-14 20:12:03 +03:00
redtea 55c37a0097 fix: the spinner by the path takes room only while busy, opening and closing smoothly 2026-10-14 20:03:03 +03:00
redtea cefdae267b fix: the spinner by the path takes only its own width 2026-10-14 19:53:42 +03:00
redtea 44e7137d86 fix: the spinner by the path keeps its place, so the path field does not jump 2026-10-14 19:51:34 +03:00
redtea 10ffa393ca fix: a closed terminal hangs its shell up instead of killing it, so the history is saved; idle shells end with their session, and on agent stop 2026-10-14 19:50:50 +03:00
redtea 114c6fc530 fix: static files linked with their content hash, so a release is never shadowed by a cached copy (Cloudflare kept the old CSS) 2026-10-14 19:40:31 +03:00
redtea 0aa066fd01 fix: a smaller frame for the properties button in the path field 2026-10-14 19:37:37 +03:00
redtea 21436d5c73 feat: the folder properties button sits inside the path field 2026-10-14 19:37:25 +03:00
redtea 65359f5497 feat: file and folder properties - mode, owner and group, applied to a folder's contents too (0.0.8) 2026-10-14 19:31:25 +03:00
redtea 8eb2bbd35d feat: close buttons as a red cross in a square; static files revalidated by content, so a release reaches the browser at once 2026-10-14 19:24:12 +03:00
redtea 55db8f03b3 chore: keep the version at 0.0.7 2026-10-14 19:07:29 +03:00
redtea e9a845a6ea feat: files go in 4 MB pieces, several at once and each retried on its own, so they pass a proxy like Cloudflare and survive a bad link (0.0.8) 2026-10-14 19:06:59 +03:00
redtea 23e382bf7f fix: the terminal opens in the user's home directory, not in / 2026-10-14 18:53:07 +03:00
redtea 438b20dae0 chore: log where and after how many bytes a file transfer breaks 2026-10-14 18:49:07 +03:00
redtea 19f8b5291e fix: no arrow on the sorted column 2026-10-14 18:46:56 +03:00
redtea c911143eb2 feat: SQLite preview read straight from the file, and sorting of the file table by column, kept per folder 2026-10-14 18:46:32 +03:00
redtea 1a90af7800 feat: a spinner on the save button while a file saves 2026-10-14 18:36:48 +03:00
redtea 92c9616826 feat: toasts show over the file viewer; icon buttons in the terminal's copy menu 2026-10-14 18:31:45 +03:00
redtea 2db44e53f7 feat: red text and border on every close button 2026-10-14 16:15:03 +03:00
redtea 68ede6ab25 feat: previews of PDF, Word, spreadsheets, CSV and HTML in the file manager; the find bar's buttons back in one row 2026-10-14 16:09:02 +03:00
redtea d448c1e3a4 fix: find and replace fields of one width, a red close button 2026-10-14 16:01:59 +03:00
redtea 3d6b04e78a fix: binary editing only for binary files and the ones shown as a preview 2026-10-14 16:00:20 +03:00
redtea e1f5595afc feat: files up to 512 MB as streams, a code editor with highlighting and find/replace, media preview, hex editing; touch selection in the terminal (0.0.7) 2026-10-14 15:52:38 +03:00
redtea 3e46116a9b feat: upload progress by the file manager spinner, and a list of the uploads behind it 2026-10-14 15:28:19 +03:00
redtea 10b8995041 feat: rename a file in place from its row in the file manager 2026-10-14 15:25:50 +03:00
redtea 225b7ecfcc fix: terminal input reaches the shell in the order it was typed 2026-10-14 15:18:14 +03:00
redtea cb2673d941 Revert "feat: DEL in place of PGDN among the terminal's extra keys"
This reverts commit cf855fdc23.
2026-10-14 14:59:18 +03:00
redtea cf855fdc23 feat: DEL in place of PGDN among the terminal's extra keys 2026-10-14 14:58:06 +03:00
redtea d456ffe757 feat: phone layout for the panel, and Termux-style extra keys in the terminal 2026-10-14 14:43:06 +03:00
redtea 81a2c96e0b fix: centre the ufw switch under the create button 2026-10-13 23:23:03 +03:00
redtea e0dc15740d feat: the ufw switch of a new forward rule goes under the button 2026-10-13 23:22:07 +03:00
redtea fbd8f30de3 feat: the new forward rule form on one line, with short type names 2026-10-13 23:18:49 +03:00
redtea 67c02ad6b4 chore: keep the version at 0.0.6 2026-10-13 23:13:15 +03:00
redtea 5155bc36cd feat: measure each direction of the speed test for 7 s (0.0.7) 2026-10-13 23:12:30 +03:00
redtea ab8751d436 feat: live reading during the speed test, and a 15 s limit that keeps what was measured 2026-10-13 23:06:56 +03:00
redtea 6441b4329d fix: don't let a slow browser link time out the relayed speed test 2026-10-13 22:49:19 +03:00
redtea 189050b2b6 fix: end the browser speed test feed without a 410 or 502 2026-10-13 22:33:42 +03:00
redtea fd75f324b8 fix: keep the speed results hidden until there is a measurement 2026-10-13 21:02:39 +03:00
redteaandClaude Opus 5.5 789e1da93b feat: open a port in ufw by hand from the panel settings
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 20:42:09 +03:00
redteaandClaude Opus 5.5 87921a1381 feat: speed test through the browser, client - panel - browser (0.0.6)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 20:31:41 +03:00
redteaandClaude Opus 5.5 b51363bf72 feat: optional label on a forward rule
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 20:31:41 +03:00
redteaandClaude Opus 5.5 201d63ee1e feat: open the file manager in the user's home directory
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 20:31:41 +03:00
redtea 5e5718b397 chore: call the machine edit link "Изменить" in page headers 2026-10-13 19:06:49 +03:00
redtea c675bb0ec7 feat: status and settings links in the header of every machine page 2026-10-13 19:06:11 +03:00
redtea a02fdc01b0 feat: show the name of an icon button on hover 2026-10-13 19:04:57 +03:00
redtea 8ba2dd1f1b chore: name the actions column, drop hardcoded tunnel port and 2 KB mentions
The tunnel port default lives only in tunnel.DefaultPort; docs and
comments refer to "the tunnel port". The file manager hint no longer
states an edit limit that was not the real one.
2026-10-13 19:04:22 +03:00
redtea 171e14176a fix: small fixes 2026-10-13 19:00:50 +03:00
redtea 86cb0226ab feat: machine status page, speed test, live presence, forwards without ufw (0.0.5)
Status page per machine: CPU load, RAM and disk of the client, sampled
by the agent only while the page is open and visible (one reading
serves every open tab), plus a button that has the agent measure its
link to the host: ping, down and up.

Online/offline now comes from the control polls the hub holds open, so
a dropped agent shows offline at once instead of 45 s later, and the
dashboard keeps the status column live.

A forward rule can be created "without ufw": the panel opens nothing in
ufw for it and so revokes nothing when it is deleted. The checkbox is
shown only while the ufw integration is on.

Dashboard row actions are icons; a status action is added.
2026-10-13 18:56:15 +03:00
redtea 7f4d645f19 fix: keep the machine row buttons on one line 2026-10-13 16:57:54 +03:00
redtea f3cb7701f2 fix: remove an agent's forward rules with the agent (0.0.4)
Deleting a machine left its forward rules behind: the host ports stayed
bound and "busy" with no page to remove them from. The rules, their
listeners and ufw rules now go with the agent, and rules orphaned by
older versions are dropped on start.

A host port equal to the direct tunnel port gets its own message.
2026-10-13 16:30:53 +03:00
redtea 328ade651c fix: fix ufw 2026-10-13 16:16:31 +03:00
redtea 5e6e0b8a3a chore: run the server unit as root from /opt/lpra
The panel's ufw integration needs root; the database now lives next to
the binary in /opt/lpra.
2026-10-13 14:55:37 +03:00
redtea bcfe909aaa feat: tunnel host and port in the panel settings, ufw kept in sync
The direct tunnel port (8446 by default) and the host agents dial for
it can be set in Settings. The server moves its listeners at once and
agents pick the address up from the panel: on connect and with every
poll, unless TUNNEL_ADDR / TUNNEL_PORT pin it locally.

ufw: run with a C locale (the output is translated), do not adopt rules
that existed before the panel, skip loopback-only listeners, re-open
what saved forwards need on start and when the integration is turned
on, move the shared tunnel rule with the port and drop it when no
direct forward is left. The last ufw error is shown in Settings.

The forwards page shows the real tunnel and panel ports.
2026-10-13 14:50:50 +03:00
redtea 08fc2d79a8 fix: take the file edit limit in the panel from the agent's read cap
The limit was repeated in the page; a file of exactly the cap is read
in full, so it is editable too.
2026-10-13 14:50:49 +03:00
redtea 7cb9da8569 chore: take the version from VERSION.txt only (0.0.3)
VERSION.txt is embedded into the server and the agent, so a plain
go build reports the real version and build.sh no longer injects it.
2026-10-13 14:50:49 +03:00
134 changed files with 25439 additions and 1059 deletions

No files matched your search

-1
View File
@@ -34,7 +34,6 @@ go.work.sum
bin
dist
*.db
.DS_Store
*/.DS_Store
+198 -32
View File
@@ -1,63 +1,229 @@
# LPRA — LongPolling Remote Access
# LPRA - LongPolling Remote Access
Лёгкий self-hosted сервис для доступа к Linux-машинам **без белого IP** (за NAT).
Агент на клиенте сам исходяще подключается к серверу по HTTP long-polling; админ
в веб-панели видит список машин и открывает по каждой **терминал в браузере** (по умолчанию от root, пользователя можно сменить).
Агент на клиенте сам исходяще подключается к серверу - по WebSocket, а где он не проходит,
по HTTP long-polling. В веб-панели видно список машин, и по каждой есть терминал в браузере,
файловый менеджер, проброс портов и статус.
```
браузер ──WS──> [nginx :8445 TLS] ──> [Go server 127.0.0.1:8091] <──long-poll── агент (root, за NAT)
│
SQLite
браузер ──HTTPS/WS──> [Cloudflare] ──> [nginx TLS] ──> [Go server 127.0.0.1:8091] <──WS / long-poll── агент (за NAT)
│ ▲
SQLite └──── прямой TCP/UDP-туннель :8446 (проброс tcp/udp)
```
## Возможности
- **Одна страница** - Машины, WorkSpace (терминалы и файлы), Настройки; переходы без перезагрузки.
Вкладки WorkSpace живут, пока открыта вкладка браузера: уход на другие страницы их не
закрывает, перезагрузка сбрасывает (браузер сначала спросит). «Порты», «Статус» и «Изменить»
машины открываются окном поверх текущей страницы - из списка машин или правым кликом
по заголовку вкладки или панели.
- **WorkSpace** - вкладки на нескольких машинах сразу, до четырёх панелей в одной вкладке -
терминал или файловый менеджер (перетаскиванием, с изменяемыми размерами), переименование
вкладок. На телефоне - панель клавиш (Esc, Ctrl, Alt, стрелки и т.д.).
- **SSH-терминал** - в меню «+»: агент открывает SSH-соединение к другому хосту (по паролю,
ключу - можно загрузить файл - или своими ключами из `~/.ssh`), шелл открывается во вкладке,
а шаги подключения видны в статусе. Ключ хоста запоминается при первом входе
(`ssh_known_hosts` рядом с `state.json` агента); если он сменится, вход откажет.
- **Файлы** - просмотр, редактор с подсветкой, hex-редактор, предпросмотр картинок, видео, PDF,
баз SQLite с правкой (значение по клику, добавление и удаление строк, вкладка SQL; агент меняет базу через сам SQLite, так что работающая база не ломается); загрузка и скачивание до 512 МБ кусками по 4 МБ (проходит через Cloudflare);
свойства (chmod/chown), zip/unzip.
- **Файлы по SSH** - в меню «+» › «Файловый менеджер» › «SSH-хост…»: машина подключается к хосту
и работает с файлами командами по SSH, SFTP не нужен. Есть всё, кроме редактора SQLite;
zip и распаковка - если на хосте установлены `zip` и `unzip`. Оборвалось подключение -
панель предложит переподключиться.
- **Перетаскивание файлов** - строку (или все отмеченные) в другую файловую панель или в папку
в ней: окно «Переместить / Копировать» у места броска, между машинами и SSH-хостами - с
прогрессом. В терминал той же машины или того же SSH-хоста - вставляется путь в кавычках,
в терминал другой - нельзя, с подсказкой почему. Задержать над заголовком вкладки - она
откроется. Только на компьютере.
- **Перемещение и копирование** - внутри машины или, кнопкой «сеть» в том же окне, **на другую
машину или на SSH-хост**: папка уходит потоком tar без промежуточных файлов, с прогрессом,
скоростью и отменой. «Переместить» удаляет исходник после успешной отправки. Правила для
существующего: сообщить / заменить / оставить как есть.
- **ИИ-ассистент** - круглая кнопка чата (на компьютере её и окно можно перетаскивать); каждый
включает себе сам в профиле. Видит вкладки, активный терминал или папку и список ваших машин,
отвечает потоком. Действия - кнопками: выполнить команду в терминале нужной машины, прочитать,
показать или записать файл - всё с правами пользователя. Плашка «Агент» - делает шаги сам,
спрашивая перед каждой командой и записью; «Думать» - сначала рассуждает (свёрнуто над
ответом). Модели задаёт администратор в «Настройки › ИИ-ассистент»: их может быть несколько, у каждой своё
название в чате (и короткое, до 12 символов, - для кнопки выбора), тип - Gradio (по умолчанию - прокси к Qwen Omni), OpenAI-совместимый API или Anthropic,
адрес, модель и ключ (хранится на сервере, в браузер не уходит). Если моделей несколько, в чате рядом с
плашками появляется выбор, первая - по умолчанию; выбор запоминается за пользователем. В «Аккаунт › Вид»
можно показывать выбор всегда.
Плашка «Поиск» даёт ассистенту искать в интернете; поисковик выбирается там же: DuckDuckGo
(по умолчанию, без настройки), свой SearXNG (нужен `search.formats: [json]`), Brave Search API
или Tavily (по ключу). Если выбранный не ответил, поиск идёт через DuckDuckGo.
**Картинки**: если администратор задал API-ключ в «Настройки › ИИ-ассистент › Генерация изображений»
([Vertex API](https://s2.x-grk.com/api/vertex/docs)), модели сами, по смыслу разговора и без плашки, рисуют картинки
и правят фото - то, что пользователь прикрепил к сообщению, или последнюю созданную картинку
(блоки ```` ```image ```` и ```` ```image edit ````, формат и качество - в первой строке; улучшение описания
можно отключить: `noenhance`). Картинка показывается в чате, описание в карточке сворачивается.
Название прикреплённого файла открывает его (или скачивает). Удаление чата стирает его содержимое, в том числе
текст вложений, из файлов базы; картинки хранятся на сервисе Vertex и оттуда не удаляются.
- **Сохранённые SSH-хосты** - в окне SSH-подключения галочка «Сохранить»: хост и вход (пароль или
ключ) хранятся на сервере, у каждого пользователя свои, зашифрованы AES-256-GCM ключом сервера.
Ключ берётся из `LPRA_SECRET_KEY` (32 байта, hex или base64) или из файла `<база>.key` рядом с
базой, который создаётся сам - храните его вместе с бэкапом базы, без него сохранённые входы
не расшифровать.
- **Проброс портов** - `http` через панель, `tcp`/`udp` через прямой туннель.
- **Статус** - нагрузка, память, диски, сеть, тест скорости канала до сервера.
- **Вход** - пароль и/или ключи доступа (passkeys); пароль можно выключить, когда есть ключ.
- **PIN-код** (необязательный, 4 цифры) - спрашивается в каждой новой вкладке браузера, но не при
перезагрузке. На устройстве можно не спрашивать его 5/15/30/60 минут после ввода.
Пять неверных попыток - выход из сессии.
## Пользователи и роли
- **Администратор** видит всё: все машины, настройки, пользователей, ufw.
- **Пользователь** видит только выданные ему машины (терминал, файлы, порты, статус, «Изменить» без удаления);
настроек, других пользователей и чужих машин для него нет.
Роль и машины задаются в «Настройки → Пользователи» (квадратная кнопка у пользователя открывает список машин).
Там же галочка «Пароль» включает или выключает вход по паролю (выключить можно, когда у человека есть ключ доступа),
а «Закрепить» не даёт ему поменять это самому. Последнего администратора нельзя удалить или понизить.
## База данных
По умолчанию - SQLite (файл из `-db`, по умолчанию `lpra.db`). В «Настройки › База данных» можно выбрать
PostgreSQL: хост, порт, база, пользователь, пароль, SSL; кнопка «Проверить и сохранить» подключается и
записывает выбор в `<база>.backend.json` рядом с SQLite-файлом (права 0600), «Перезапустить сервер»
завершает процесс - systemd (`Restart=always`) поднимает его уже на PostgreSQL. При первом запуске на
пустой PostgreSQL все данные копируются из SQLite-файла, сам файл остаётся как был (к нему можно
вернуться тем же выбором). История ИИ-ассистента тоже хранится в базе - у каждого пользователя
последние 200 сообщений.
## Компоненты
- **Сервер** (`cmd/server`) — админ-панель + точка встречи для агентов. За nginx (TLS).
- **Агент** (`cmd/agent`) — Linux-бинарь, systemd-сервис от root, PTY по запросу.
- **install.sh** — интерактивный установщик, отдаётся сервером как статика.
- **Сервер** (`cmd/server`) - панель и точка встречи для агентов. Ставится за nginx (TLS).
- **Агент** (`cmd/agent`) - бинарь для Linux (amd64/arm64), systemd-сервис от root.
- **install.sh** - установщик агента, отдаётся сервером.
## Сборка
```sh
./build.sh
```
Получаются `bin/lpra-server` и `dist/agent-linux-{amd64,arm64}`.
Получаются `bin/lpra-server*` и `dist/agent-linux-{amd64,arm64}`.
Версия задаётся в одном месте - `VERSION.txt`, она вшивается в сервер и агент.
## Установка сервера одной командой
На Linux-сервере (amd64 или arm64, systemd):
```sh
wget -qO- https://gitlab.com/itzredtea/lpra/-/raw/main/server-install.sh | sudo sh
```
Скрипт `server-install.sh`:
- качает бинарник сервера под архитектуру машины из [последнего релиза](https://gitlab.com/itzredtea/lpra/-/releases/permalink/latest)
и папку `dist/` (бинарники агентов) из ветки `main`, кладёт всё в `/opt/lpra`;
- ставит службу systemd `lpra-server` (порт спросит, по умолчанию `8091`);
- по желанию настраивает nginx: спросит домен, создаст сайт на порт 80 (при необходимости поставит nginx),
затем `certbot --nginx` - если certbot с плагином nginx не установлен, этот шаг пропускается;
- если включён ufw - предложит открыть 80, 443 и туннель `8446` tcp/udp;
- запускает службу, перезагружает nginx и проверяет, что сервер отвечает.
Вопросы задаются в терминале, даже когда скрипт идёт через `| sudo sh`. Без терминала ответы берутся
из переменных: `LPRA_PORT`, `LPRA_NGINX` (y/n), `LPRA_DOMAIN`, `LPRA_EMAIL`, `LPRA_CERTBOT` (y/n), `LPRA_UFW` (y/n):
```sh
wget -qO- https://gitlab.com/itzredtea/lpra/-/raw/main/server-install.sh | sudo LPRA_NGINX=y LPRA_DOMAIN=lpra.example.com sh
```
Повторный запуск той же командой - **обновление**: бинарник и `dist/` заменяются, база (`/opt/lpra/lpra.db`
и её ключ `lpra.db.key`), настройки службы и nginx остаются. После установки откройте `/setup` - мастер
создаст администратора.
## Запуск сервера
```sh
./bin/lpra-server -addr 127.0.0.1:8091 -dist ./dist -db ./lpra.db
```
Поставьте перед ним nginx по `deploy/nginx.conf.example` (TLS, long-poll/WS).
Локально без TLS можно добавить `-insecure-cookies`.
| Флаг | Переменная | По умолчанию | |
|---|---|---|---|
| `-addr` | `LPRA_ADDR` | `127.0.0.1:8091` | адрес HTTP (за nginx) |
| `-db` | `LPRA_DB` | `lpra.db` | база SQLite |
| `-dist` | `LPRA_DIST` | `dist` | бинарники агента для `/download/` |
| `-tunnel` | `LPRA_TUNNEL_ADDR` | `:8446` | прямой туннель TCP+UDP (пусто - выключен) |
| `-insecure-cookies` | `LPRA_INSECURE_COOKIES=1` | | куки без Secure - только для локального запуска без TLS |
Первый запуск: откройте `/setup` — мастер создаст администратора. Дальше `/settings`:
адрес хоста (по умолчанию `https://localhost:8445`), enroll-секрет и готовая
команда установки.
Пример юнита - `deploy/lpra-server.service`.
## Установка агента на клиент
Одной командой (под root):
```sh
curl -fsSL https://localhost:8445/install.sh | sudo HOST=https://localhost:8445 ENROLL_SECRET=<секрет> sh
Первый запуск: откройте `/setup` - мастер создаст администратора. Дальше в `/settings`:
адрес хоста, enroll-секрет, короткий код установки и готовая команда.
### nginx
Конфиг - `deploy/nginx.conf.example`. Обязательно:
- заголовки `Upgrade` / `Connection` (через `map $http_upgrade $connection_upgrade`) - без них
WebSocket не пройдёт, и агенты и терминалы останутся на long-polling (работает, но медленнее
на больших выводах);
- `proxy_read_timeout` больше 25 с (удержание long-poll), `proxy_buffering off`.
### Прямой адрес (без Cloudflare)
В «Настройки → Адрес хоста» можно задать второй, прямой адрес - например `https://direct.lpra.example.com`,
DNS-запись без проксирования. Тогда:
- агенты, не достучавшись до основного адреса 30 секунд, переходят на прямой и каждые 5 минут проверяют,
не ожил ли основной; уже установленные агенты получают прямой адрес с сервера сами (он хранится в `state.json`);
- установщик берёт адреса с того домена, с которого его скачали: с основного - агент работает через основной
с резервом на прямой, с прямого - наоборот (и бинарник качается с прямого); если первый адрес не отвечает,
бинарник качается со второго. В «Полных вариантах» есть короткая команда и через прямой адрес;
- в «Изменить → Связь с сервером» у клиента, подключённого через прямой адрес, после статуса стоит «(direct)»;
- если хост туннеля не задан, агентам отдаётся хост прямого адреса;
- ключи доступа основного домена работают и на его поддомене.
### Cloudflare
Работает за проксирующим Cloudflare: WebSocket он пропускает, файлы идут кусками меньше лимита
на запрос, статика с хешем в имени кэшируется. Чтобы панель видела настоящие IP (ограничение
попыток входа и кода установки считается по адресу), включите в nginx real IP:
```nginx
real_ip_header CF-Connecting-IP;
set_real_ip_from 173.245.48.0/20; # и остальные диапазоны с https://www.cloudflare.com/ips/
```
Или интерактивно (спросит секрет, а если агент уже стоит — предложит удалить):
Прямой туннель (`:8446`) - это не HTTP, через проксирующий Cloudflare он не идёт: в
«Настройки → Прямой туннель» укажите для агентов IP хоста или DNS-имя без проксирования.
## Установка агента
Короткой командой из `/settings` (код из 3-24 букв и цифр, кнопка «Новый код» меняет его):
```sh
curl -fsSL https://localhost:8445/install.sh | sudo sh
curl -fsSL https://panel.example/i/<код> | sudo sh
```
Или полной:
```sh
curl -fsSL https://panel.example/install.sh | sudo HOST=https://panel.example ENROLL_SECRET=<секрет> sh
```
Без параметров установщик спросит секрет, а если агент уже стоит - предложит удалить.
Обновить агента можно кнопкой в панели («Изменить» → обновить).
## Связь агента с сервером
В «Изменить» → «Связь с сервером» для каждой машины:
- **auto** (по умолчанию) - WebSocket; если он не подключается, long-polling ровно минуту, потом
снова попытка WebSocket;
- **ws** - только WebSocket;
- **lp** - только long-polling.
## Проброс портов и прямой туннель
Правила создаются на странице машины «Порты». Режим `http` (в панели - http-lp) идёт через
соединение агента с панелью: по WebSocket, а если он недоступен - по long-polling;
`tcp`/`udp` - через прямой туннель: агент держит исходящее соединение к порту туннеля хоста,
по нему мультиплексируются все соединения.
Порт туннеля и хост для агентов меняются в `/settings` → «Прямой туннель»; агенты забирают их
сами, переустановка не нужна. Локальные `TUNNEL_ADDR` / `TUNNEL_PORT` в окружении агента имеют
приоритет (`TUNNEL_ADDR=off` отключает туннель).
Если на хосте активен ufw и сервер запущен от root, панель сама открывает порты правил и порт
туннеля, а при удалении правила закрывает. При `ProtectSystem=strict` в юните нужно
`ReadWritePaths=/etc/ufw /run`.
## Пользователь терминала
По умолчанию терминал открывается от пользователя, под которым запущен агент (на Linux — root).
В окне «Изменить машину» можно выбрать другого пользователя машины или создать нового
(имя, пароль, опционально — добавить в sudo). Смена и создание работают, только если агент запущен от root.
Файловый менеджер по-прежнему работает от пользователя агента.
По умолчанию терминал открывается от пользователя агента (root). В «Изменить машину» можно выбрать
другого пользователя или создать нового (с паролем, по желанию с sudo) - если агент запущен от root.
Файловый менеджер работает от пользователя агента.
## Безопасность
- Пароли админов — bcrypt; сессии в HttpOnly/Secure/SameSite куках; CSRF на формах; rate-limit логина.
- Агенты: общий enroll-секрет → индивидуальный токен (хранится хешем, сверка constant-time).
- Открытие терминала пишется в таблицу `audit` (терминал даёт root-доступ), как и смена/создание пользователя.
- Пароли админов - bcrypt; сессии в HttpOnly/Secure/SameSite куках; CSRF на формах;
ограничение попыток входа, PIN и кода установки.
- Ключи доступа (WebAuthn) - вход без пароля; пароль можно выключить совсем.
- Агенты: общий enroll-секрет → индивидуальный токен (хранится хешем, сверка constant-time);
туннель авторизуется HMAC от токена.
- SSH-пароли и ключи передаются агенту один раз для подключения и нигде не сохраняются.
- Открытие терминала и SSH, перенос файлов на другие машины, смена пользователя пишутся в `audit`.
## Скриншоты
<img width="1256" height="709" alt="image" src="https://github.com/user-attachments/assets/5f0b30f1-d5e3-4b62-817e-4041641df6e3" />
<img width="1256" height="709" alt="image" src="docs/screenshots/1.png" />
<img width="1254" height="532" alt="image" src="https://github.com/user-attachments/assets/0207a7f0-2b59-40ee-abd1-d8673059cc24" />
<img width="1254" height="532" alt="image" src="docs/screenshots/2.png" />
<img width="1255" height="526" alt="image" src="https://github.com/user-attachments/assets/4dcbe907-33f5-493e-a56f-01cd8bf0fea1" />
<img width="1255" height="526" alt="image" src="docs/screenshots/3.png" />
<img width="1260" height="654" alt="image" src="https://github.com/user-attachments/assets/fdf59733-8e5d-46e3-b232-235ddfe1be73" />
<img width="1260" height="654" alt="image" src="docs/screenshots/4.png" />
+1 -1
View File
@@ -1 +1 @@
0.0.2
0.2.4
+3 -4
View File
@@ -5,10 +5,9 @@ set -eu
cd "$(dirname "$0")"
VERSION="${VERSION:-$(cat VERSION.txt 2>/dev/null || echo 0.0.0)}"
PKG="github.com/x-grk/longpolling-remote-access"
LDFLAGS="-s -w -X $PKG/internal/server.Version=$VERSION -X $PKG/internal/agent.Version=$VERSION"
echo ">> version $VERSION"
# The version comes from VERSION.txt, which is embedded into every binary.
LDFLAGS="-s -w"
echo ">> version $(cat VERSION.txt)"
mkdir -p dist bin
+16 -5
View File
@@ -10,19 +10,25 @@ import (
"net/url"
"os"
"os/signal"
"strconv"
"strings"
"syscall"
"github.com/x-grk/longpolling-remote-access/internal/agent"
"github.com/x-grk/longpolling-remote-access/internal/tunnel"
)
func main() {
host := envOr("HOST", "https://REDACTED:8445")
cfg := agent.Config{
Host: host,
Fallback: strings.TrimRight(os.Getenv("FALLBACK_HOST"), "/"),
EnrollSecret: os.Getenv("ENROLL_SECRET"),
Name: os.Getenv("AGENT_NAME"),
StatePath: envOr("STATE_PATH", "/etc/lpagent/state.json"),
TunnelAddr: tunnelAddr(host),
// A locally set address wins over the one from the panel settings.
TunnelPinned: os.Getenv("TUNNEL_ADDR") != "" || os.Getenv("TUNNEL_PORT") != "",
}
if cfg.EnrollSecret == "" {
// Only required for the very first run (before enrollment).
@@ -34,7 +40,11 @@ func main() {
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
defer stop()
if err := agent.New(cfg).Run(ctx); err != nil {
err := agent.New(cfg).Run(ctx)
// Stopping (an update, a restart): hang the terminals up, so their shells
// save their history instead of being killed.
agent.CloseTerminals()
if err != nil {
log.Fatalf("agent: %v", err)
}
}
@@ -46,9 +56,10 @@ func envOr(key, def string) string {
return def
}
// tunnelAddr derives the direct-tunnel address (host:8446) from the server URL.
// TUNNEL_ADDR overrides it entirely; TUNNEL_PORT overrides just the port.
// Set TUNNEL_ADDR=off to disable the direct tunnel.
// tunnelAddr derives the direct-tunnel address (host:port) from the server URL.
// It is only the starting point: the agent follows the host and port set in
// the panel settings, unless TUNNEL_ADDR (the whole address) or TUNNEL_PORT
// (just the port) pins it here. Set TUNNEL_ADDR=off to disable the tunnel.
func tunnelAddr(hostURL string) string {
if v := os.Getenv("TUNNEL_ADDR"); v != "" {
if v == "off" {
@@ -60,5 +71,5 @@ func tunnelAddr(hostURL string) string {
if err != nil || u.Hostname() == "" {
return ""
}
return net.JoinHostPort(u.Hostname(), envOr("TUNNEL_PORT", "8446"))
return net.JoinHostPort(u.Hostname(), envOr("TUNNEL_PORT", strconv.Itoa(tunnel.DefaultPort)))
}
+4 -2
View File
@@ -10,21 +10,23 @@ import (
"log"
"net/http"
"os"
"strconv"
"time"
"github.com/x-grk/longpolling-remote-access/internal/server"
"github.com/x-grk/longpolling-remote-access/internal/tunnel"
)
func main() {
addr := flag.String("addr", envOr("LPRA_ADDR", "127.0.0.1:8091"), "listen address (behind nginx)")
dbPath := flag.String("db", envOr("LPRA_DB", "lpra.db"), "path to SQLite database")
distDir := flag.String("dist", envOr("LPRA_DIST", "dist"), "directory with agent binaries served at /download/")
tunnelAddr := flag.String("tunnel", envOr("LPRA_TUNNEL_ADDR", ":8446"), "listen address for the direct TCP+UDP forward tunnel (empty disables)")
tunnelAddr := flag.String("tunnel", envOr("LPRA_TUNNEL_ADDR", ":"+strconv.Itoa(tunnel.DefaultPort)), "listen address for the direct TCP+UDP forward tunnel (empty disables)")
insecureCookies := flag.Bool("insecure-cookies", envOr("LPRA_INSECURE_COOKIES", "") == "1",
"do not mark cookies Secure (plain-HTTP local testing only)")
flag.Parse()
db, err := server.OpenDB(*dbPath)
db, err := server.OpenStore(*dbPath)
if err != nil {
log.Fatalf("open db: %v", err)
}
+17 -20
View File
@@ -1,24 +1,24 @@
# systemd unit for the LPRA host (server).
#
# Unlike the agent, the server does NOT need root: it listens on a high local
# port (127.0.0.1:8091) behind nginx, so run it as an unprivileged user.
# The server runs as root so that the panel can open forwarded ports in ufw;
# it listens on a high local port (127.0.0.1:8091) behind nginx.
#
# Install:
# sudo useradd --system --home /opt/lpra --shell /usr/sbin/nologin lpra
# sudo mkdir -p /opt/lpra /var/lib/lpra
# sudo mkdir -p /opt/lpra
# sudo cp bin/lpra-server /opt/lpra/
# sudo cp -r dist /opt/lpra/dist # agent binaries served at /download/
# sudo chown -R lpra:lpra /opt/lpra /var/lib/lpra
# sudo cp deploy/lpra-server.service /etc/systemd/system/
# sudo systemctl daemon-reload && sudo systemctl enable --now lpra-server
#
# Config overrides can go in /etc/lpra-server.env (see EnvironmentFile below),
# e.g. LPRA_ADDR=127.0.0.1:8091
# Direct port forwarding (tcp/udp modes) uses the tunnel port on the host, which
# has to be open in the firewall for tcp and udp. (The "http" forward mode and
# terminal/files go through nginx :8445 and need nothing extra.) Set the tunnel
# address with LPRA_TUNNEL_ADDR, or set it empty to disable.
# The port, and the host agents dial for it, can also be changed in the panel
# settings; agents pick the new address up from the panel by themselves.
#
# Direct port forwarding (tcp/udp modes) uses port 8446 on the host — open it in
# the firewall: ufw allow 8446/tcp ; ufw allow 8446/udp
# (The "http" forward mode and terminal/files go through nginx :8445 and need
# nothing extra.) Override with LPRA_TUNNEL_ADDR, or set it empty to disable.
# ufw integration (the panel opening forwarded ports itself) needs root and a
# writable /etc/ufw and /run - see ReadWritePaths below.
[Unit]
Description=LPRA remote access host (admin panel + agent rendezvous)
@@ -26,17 +26,11 @@ After=network-online.target
Wants=network-online.target
[Service]
Type=simple
User=lpra
Group=lpra
WorkingDirectory=/opt/lpra
# Optional environment overrides; "-" means the file is not required.
EnvironmentFile=-/etc/lpra-server.env
ExecStart=/opt/lpra/lpra-server \
-addr 127.0.0.1:8091 \
-db /var/lib/lpra/lpra.db \
-db /opt/lpra/lpra.db \
-dist /opt/lpra/dist
Restart=always
@@ -51,8 +45,11 @@ PrivateDevices=true
ProtectKernelTunables=true
ProtectControlGroups=true
RestrictSUIDSGID=true
# The server only needs to write its SQLite database here.
ReadWritePaths=/var/lib/lpra
# The SQLite database, plus what ufw writes: its rules in /etc/ufw and the
# ufw/xtables locks in /run. Without the last two "ufw status" works but
# "ufw allow" fails with "'/etc/ufw/user.rules' is not writable".
ReadWritePaths=/opt/lpra /etc/ufw /run
ReadWritePaths-=/etc/ufw /run
[Install]
WantedBy=multi-user.target
+51
View File
@@ -0,0 +1,51 @@
# nginx reverse proxy for the LPRA server.
# TLS is terminated here; the Go server listens on 127.0.0.1:8091.
#
# Put this in /etc/nginx/sites-available/lp_agent and symlink it into
# sites-enabled, then: sudo nginx -t && sudo systemctl reload nginx
#
# Agents connect by WebSocket (from 0.1.0), with HTTP long polling to fall
# back on; the browser uses plain HTTP. The settings that matter:
# - Upgrade / Connection headers -> let the agents' WebSocket through
# - proxy_buffering off -> deliver long-poll responses immediately
# - proxy_read_timeout 300s -> longer than the 25s long-poll hold
#
# NOTE: proxy_pass MUST start with "http://". A bare "proxy_pass 127.0.0.1:8091;"
# causes nginx error: [emerg] invalid URL prefix.
# "Connection: upgrade" for a WebSocket, "close" for everything else.
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
server {
listen 80;
server_name example.ru;
client_max_body_size 120m;
location / {
proxy_pass http://127.0.0.1:8091;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# Agents' WebSocket.
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
# Long-poll behaviour: no buffering, generous read timeout.
proxy_buffering off;
proxy_cache off;
# Pass uploads on as they arrive: the speed test through the browser
# times them on the client.
proxy_request_buffering off;
proxy_read_timeout 300s;
proxy_send_timeout 300s;
}
}
@@ -4,25 +4,35 @@
# Put this in /etc/nginx/sites-available/lp_agent and symlink it into
# sites-enabled, then: sudo nginx -t && sudo systemctl reload nginx
#
# Everything (agent + browser terminal) is plain HTTP long-polling, so NO
# WebSocket config is needed. The two settings that matter:
# Agents connect by WebSocket (from 0.1.0), with HTTP long polling to fall
# back on; the browser uses plain HTTP. The settings that matter:
# - Upgrade / Connection headers -> let the agents' WebSocket through
# - proxy_buffering off -> deliver long-poll responses immediately
# - proxy_read_timeout 120s -> longer than the 25s long-poll hold
# - proxy_read_timeout 300s -> longer than the 25s long-poll hold
#
# NOTE: proxy_pass MUST start with "http://". A bare "proxy_pass 127.0.0.1:8091;"
# causes nginx error: [emerg] invalid URL prefix.
# "Connection: upgrade" for a WebSocket, "close" for everything else.
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
server {
listen 8445 ssl;
server_name REDACTED;
listen 443 ssl;
server_name example.ru;
ssl_certificate /etc/letsencrypt/live/REDACTED/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/REDACTED/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
# Agent binaries can be large; allow the download.
client_max_body_size 64m;
# Files go through the panel in 4 MB pieces; 64m leaves room for a client
# that predates the pieces, which takes a file of up to 32 MB in one request.
client_max_body_size 120m;
location / {
proxy_pass http://127.0.0.1:8091;
@@ -32,11 +42,17 @@ server {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# Agents' WebSocket.
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
# Long-poll behaviour: no buffering, generous read timeout.
proxy_buffering off;
proxy_cache off;
proxy_read_timeout 120s;
proxy_send_timeout 120s;
# Pass uploads on as they arrive: the speed test through the browser
# times them on the client.
proxy_request_buffering off;
proxy_read_timeout 300s;
proxy_send_timeout 300s;
}
}
Vendored Executable
BIN
View File
Binary file not shown.
Vendored Executable
BIN
View File
Binary file not shown.
+144
View File
@@ -0,0 +1,144 @@
# Перенос проекта с GitHub на GitLab с сохранением истории и дат
Инструкция для локального репозитория, когда GitHub недоступен (аккаунт заблокирован) и
переносить приходится из своей копии. Так был перенесён `lpra`: 195 коммитов, 8 тегов,
3 релиза - хэши и даты совпали.
## Что сохраняется, а что нет
| Что | Сохраняется? |
|---|---|
| Даты коммитов (автора и коммита) | **Да, точно.** Они хранятся внутри коммита; push той же истории не меняет ни одного хэша |
| Ветки и теги | Да, если они есть локально (см. шаг 2) |
| Дата создания проекта | **Нет.** На gitlab.com она всегда - день создания проекта, задним числом не ставится. Начало проекта видно по первому коммиту |
| Релизы | Создаются заново, **с исходной датой** (`--released-at`); файлы релизов - заново собрать или взять свои копии |
| Issues, PR, звёзды, Actions | Нет: при заблокированном GitHub их не выгрузить |
**Нельзя:** `rebase`, `squash`, `filter-branch`, `filter-repo`, `commit --amend` по старым
коммитам, «Import from GitHub» с чужой копии - всё это переписывает коммиты и их даты.
## 0. Один раз: вход в GitLab
```sh
brew install glab
glab auth login --hostname gitlab.com --web --git-protocol https
```
Вход через браузер даёт права и на push, и на релизы. Git берёт пароль у glab - для каждого
репозитория это настраивается в шаге 4 (или один раз для всех:
`git config --global credential.https://gitlab.com.helper '!glab auth git-credential'`).
SSH-ключ тоже подойдёт, но в GitLab у ключа должен быть **Usage type: Authentication & Signing**
(с типом «Signing» по SSH не пускает).
## 1. Пустой проект в GitLab
https://gitlab.com/projects/new → **Create blank project** → имя, владелец `itzredtea`.
**Снять галочку «Initialize repository with a README»** - иначе в проекте появится свой первый
коммит, и историю придётся склеивать.
Или из терминала:
```sh
glab repo create itzredtea/ИМЯ --public # или --private; README не создаётся
```
## 2. Проверить, что всё есть локально
```sh
cd ~/CodeRepos/ИМЯ
git status # незакоммиченное закоммитить или убрать
git branch -a # ветки: свои и remotes/origin/*
git tag -l # теги
```
- Ветки, которых нет локально, но есть `remotes/origin/X`, тоже переносятся (шаг 3).
- **Теги, созданные на GitHub при публикации релиза, могут не быть локально** - если их не
делали `git fetch --tags`. Такой тег создаётся заново на нужном коммите (шаг 5). Коммит
ищется по дате публикации релиза: последний коммит перед ней
(`git log --format='%h %cd %s' --date=iso --until='ДАТА'`).
Снимок до переноса - для сверки:
```sh
git log --branches --remotes=origin --tags --format='%H %aI %cI' | sort > /tmp/before.txt
git show-ref --tags | sort > /tmp/tags-before.txt
```
## 3. Отправить историю
```sh
git remote rename origin github # GitHub остаётся под своим именем
git remote add origin https://gitlab.com/itzredtea/ИМЯ.git
H='!glab auth git-credential'
git -c credential.helper= -c "credential.helper=$H" push origin --all # все локальные ветки
git -c credential.helper= -c "credential.helper=$H" push origin 'refs/remotes/github/*:refs/heads/*' # ветки, что были только на GitHub
git -c credential.helper= -c "credential.helper=$H" push origin --tags
```
Вторая команда push может сказать про `HEAD` - это не ветка, сообщение можно не смотреть.
## 4. Сверить и переключиться
Сверка - по свежей копии из GitLab:
```sh
rm -rf /tmp/check && git clone -q https://gitlab.com/itzredtea/ИМЯ.git /tmp/check
cd /tmp/check && git log --remotes=origin --tags --format='%H %aI %cI' | sort > /tmp/after.txt
git show-ref --tags | sort > /tmp/tags-after.txt
diff /tmp/before.txt /tmp/after.txt && echo "коммиты и даты совпали"
diff /tmp/tags-before.txt /tmp/tags-after.txt && echo "теги совпали"
```
Пустой `diff` - история перенесена один в один. Дальше работать с GitLab:
```sh
cd ~/CodeRepos/ИМЯ
git config --local credential.https://gitlab.com.helper '!glab auth git-credential'
git fetch origin && git branch -u origin/main main
```
## 5. Релизы с исходными датами
Дата - время публикации релиза на GitHub (UTC). Файлы - свои копии или сборка из коммита
релиза во временной папке:
```sh
git worktree add --detach /tmp/rel-1.2.0 1.2.0 # тег или хэш коммита релиза
cd /tmp/rel-1.2.0 && ./build.sh # как проект собирается
glab release create 1.2.0 bin/файл1 bin/файл2 \
-R itzredtea/ИМЯ --name "Release 1.2.0" \
--released-at "2026-10-15T16:34:53Z" \
--notes "Перенесено с GitHub, дата публикации исходная."
# тега нет (его делал GitHub) - добавить: --ref <полный хэш коммита>
git worktree remove /tmp/rel-1.2.0
```
Создавать по порядку, от старых к новым. Последним GitLab считает релиз с самой поздней
датой. Ссылки на файлы:
- конкретный релиз: `https://gitlab.com/itzredtea/ИМЯ/-/releases/ТЕГ/downloads/ФАЙЛ`
- последний: `https://gitlab.com/itzredtea/ИМЯ/-/releases/permalink/latest/downloads/ФАЙЛ`
## 6. Ссылки в самом проекте
Найти и заменить всё, что смотрит на GitHub:
```sh
grep -rn "github.com\|githubusercontent" --exclude-dir=.git .
```
| GitHub | GitLab |
|---|---|
| `https://raw.githubusercontent.com/USER/ИМЯ/main/ПУТЬ` | `https://gitlab.com/itzredtea/ИМЯ/-/raw/main/ПУТЬ` |
| `https://github.com/USER/ИМЯ/releases/latest/download/ФАЙЛ` | `https://gitlab.com/itzredtea/ИМЯ/-/releases/permalink/latest/downloads/ФАЙЛ` |
| `https://api.github.com/repos/USER/ИМЯ/releases/latest` (`tag_name`) | `https://gitlab.com/api/v4/projects/itzredtea%2FИМЯ/releases/permalink/latest` (`tag_name`) |
| `https://api.github.com/repos/USER/ИМЯ/contents/ПАПКА?ref=main` | `https://gitlab.com/api/v4/projects/itzredtea%2FИМЯ/repository/tree?path=ПАПКА&ref=main` (поле `path`) |
| картинки `github.com/user-attachments/...` | скачать, пока открываются, в `docs/` и ссылаться на файлы |
Путь Go-модуля (`module github.com/...` в `go.mod`) можно не трогать: это только имя.
## 7. После переноса
- Описание проекта в GitLab: «Перенесён с GitHub, начат ДД.ММ.ГГГГ» - раз дата создания новая.
- Токены, которые светились в чатах и логах, - отозвать и выпустить новые.
+73
View File
@@ -0,0 +1,73 @@
# План: управление потоком в прямом туннеле (tcp-проброс)
Статус: **отложено** - сделать, если на практике упрёмся. Записано 2026-10-16.
## Что сейчас
Прямой туннель (`:8446`, режим проброса `tcp`) - одно TCP-соединение на агента, по нему
мультиплексируются все проброшенные соединения этого агента кадрами
`type(1) | stream(4) | length(4) | payload` (`internal/tunnel/frame.go`).
Ёмкости хватает с запасом: на агента - одно соединение и одна горутина чтения, на проброшенное
соединение - горутина и буфер 32 КБ; шифрования в туннеле нет. 20 агентов и тысячи соединений -
десятки МБ памяти и почти ноль CPU. Потолок - сеть хоста (1 Гбит/с), через которую весь трафик
идёт дважды (пользователь ↔ хост ↔ агент).
## Проблема: блокировка всего туннеля агента одним соединением
Управления потоком (окон, как в HTTP/2 или yamux) нет, поэтому внутри одного агента медленное
соединение тормозит остальные:
- **Хост** (`internal/server/tunnelserver.go`, `agentTunnel.readLoop`): кадр `TypeData` пишется
пользователю синхронно, `c.Write(f.Payload)`, без дедлайна. Клиент, который медленно читает или
завис, останавливает чтение туннеля - встают все пробросы этого агента, пока клиент не отвалится.
- **Агент** (`internal/agent/tunnel.go`, `tunClient.readLoop`): данные кладутся в канал потока
`in` (256 кусков, до ~8 МБ). Если локальный сервис не читает и канал полон, `readLoop`
блокируется на `s.in <- f.Payload` - встаёт весь туннель агента.
Между агентами влияния нет: у каждого свой `readLoop`.
Когда проявится: через одного агента одновременно большая отдача медленному клиенту и что-то
интерактивное (SSH, веб) - интерактивное подвисает. Худший случай - зависший клиент держит
туннель агента до разрыва TCP.
## Предлагаемое решение
1. **Своя очередь записи у каждого потока** - на хосте и на агенте. `readLoop` только кладёт
кадр в очередь потока и никогда не пишет в сокет сам; пишет отдельная горутина потока.
2. **Окно (кредиты) на поток.** Новый тип кадра `TypeWindow` (stream, uint32 байт): получатель
сообщает, сколько ещё байт готов принять. Отправитель не шлёт `TypeData` сверх окна.
Начальное окно - 256 КБ; получатель возвращает кредит по мере записи в сокет (например, когда
записана половина окна). Так очередь потока ограничена окном, а медленный поток тормозит
только себя.
3. **Дедлайны записи.** Запись в пользовательский или локальный сокет - с `SetWriteDeadline`
(например, 60 с без прогресса). Не успел - поток закрывается (`TypeClose`), туннель живёт.
4. **Справедливость отправки.** Отправка в туннель уже сериализована (`wmu`); кадры данных
ограничить 32 КБ (сейчас так и есть), чтобы большой поток не занимал канал надолго.
## Совместимость со старыми агентами
Рукопожатие `LPRA-TUN1` не меняем: старый агент отвергает другое приветствие, а старый сервер -
ответ не из двух полей. Вместо этого:
- сервер знает версию агента (из БД, `versionAtLeast`) и, если агент новый, сразу после
рукопожатия шлёт кадр `TypeHello` (флаги: `flowControl`);
- агент включает окна, только получив `TypeHello`. Старый сервер его не шлёт - новый агент
работает с ним по-старому. Старому агенту сервер `TypeHello` не шлёт и окна не включает (а
пришедший неизвестный кадр старый агент и так пропустил бы: в его `readLoop` нет `default`).
Новые типы кадров: `TypeHello = 5`, `TypeWindow = 6`.
## Тесты
- Медленный читатель на одном потоке не задерживает другой поток того же агента (замер задержки
эха на втором потоке при забитом первом).
- Зависший клиент закрывается по дедлайну, туннель и остальные потоки живы.
- Новый сервер + старый агент и старый сервер + новый агент - проброс работает, как раньше.
- Пропускная способность одного потока не падает заметно (окно 256 КБ на RTT до 50 мс даёт
~40 Мбит/с на поток; если мало - окно больше или автоподстройка).
## Объём
Сервер: `tunnelserver.go` (~150 строк), агент: `tunnel.go` (~150 строк), `frame.go` (типы), тесты.
Нужна новая версия агента; панель после обновления помечает старых как устаревших.
Binary file not shown.

After

Width:  |  Height:  |  Size: 143 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 97 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 156 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 226 KiB

File diff suppressed because it is too large. Load diff
@@ -0,0 +1,159 @@
# LPRA 0.2.0 — рабочее пространство (подпроект 1)
Дата: 2026-10-15. Статус: спецификация утверждена.
Подпроект 2 (встроенный ИИ-помощник) — отдельная спецификация, делается после этого.
## Цель
Работа с несколькими машинами в одном месте, без потери открытых терминалов при переходах по панели.
Терминалы, файловые менеджеры (своих машин и по SSH) живут во вкладках одного рабочего пространства;
файлы перетаскиваются между панелями и в терминал.
Критерии готовности:
- переход Машины ↔ Терминал ↔ Настройки не обрывает сессии терминалов и не сбрасывает вкладки;
- перезагрузка страницы сбрасывает пространство (с предупреждением браузера, если есть живые сессии);
- файловый менеджер — тип панели во вкладке, в том числе по SSH;
- перетаскивание файлов: между файловыми панелями (окно «переместить / копировать»), в терминал той же машины (вставка путей), в терминал другой машины — запрет с сообщением.
## Решения, принятые в обсуждении
| вопрос | решение |
|---|---|
| Страницы «Порты», «Статус», «Изменить» | окна поверх текущей страницы |
| Файлы рядом с терминалами в одной вкладке | да: любая из до 4 панелей — терминал, SSH-терминал, файлы, SSH-файлы |
| Перезагрузка при живых сессиях | вопрос браузера «Покинуть сайт?» |
| Архитектура | одностраничная оболочка, остальное — HTML-фрагменты с сервера (вариант A) |
| Файлы по SSH | без SFTP-библиотеки: команды на хосте через SSH, как у существующей передачи tar |
| Состояние | одно на вкладку браузера, в памяти страницы; не сохраняется при перезагрузке |
## 1. Каркас и маршруты
**Оболочка.** Адреса `/`, `/terminal`, `/settings` отдают один шаблон `app.html`: шапка (LPRA, «Машины · Терминал · Настройки» — «Настройки» только администраторам, статус активной панели, аккаунт) и три области `<section data-view="machines|terminal|settings">`.
Старые адреса `/agents/{id}/terminal` и `/agents/{id}/files` отдают ту же оболочку и при загрузке создают вкладку нужного типа для этой машины (закладки не ломаются).
**Роутер** (`static/app.js`, без библиотек):
- перехватывает клики по внутренним ссылкам и `popstate`; меняет видимую область через `history.pushState`, подсвечивает пункт шапки;
- область «Терминал» создаётся один раз и потом только скрывается (`hidden`) — сессии, WebSocket и xterm живут, пока жива вкладка браузера;
- «Машины» и «Настройки» при каждом показе подгружаются заново как фрагмент: `GET /?frag=1`, `GET /settings?frag=1`.
**Фрагменты на сервере.** `render()` получает режим фрагмента (параметр `frag=1`): шаблон отдаёт только содержимое, без `<html>`, шапки и подключений. Общая разметка `head` и `topbar`, сейчас скопированная в каждый шаблон, выносится в `{{define}}`-блоки.
**Формы во фрагментах** (`app.js`, перехват `submit` внутри областей и окон):
1. `onsubmit`-обработчики (`confirm(...)`) срабатывают как обычно; отказ — ничего не происходит;
2. форма отправляется через `fetch` (`redirect: 'follow'`, тело — `FormData`);
3. итоговый адрес ответа определяет, что перерисовать: тот же фрагмент по этому адресу (сохраняются `?err=...`, `?updated=1`, `#users`);
4. формы на `/logout` и ссылки наружу работают обычной загрузкой страницы.
**Скрипты фрагментов.** Встроенные `<script>` во фрагментах не исполняются. Поведение страниц оформляется модулями с `mount(root)` / `unmount()`, которые вызывает роутер: копирование команд и коды установки (настройки), `users.js`, список машин с опросом присутствия.
`pin.js`, `scroll.js`, `account.js` подключаются один раз в оболочке.
**Предупреждение при уходе.** `beforeunload` спрашивает «Покинуть сайт?», только если в рабочем пространстве есть живая сессия терминала или SSH-терминала.
## 2. Рабочее пространство и типы панелей
**Модель.** Вкладка содержит 1–4 панели в существующих раскладках (`1`, `h`, `v`, сетка), с изменением размеров и перетаскиванием панелей и вкладок, как сейчас. Тип панели:
| тип | что это |
|---|---|
| `term` | терминал на машине (как сейчас) |
| `ssh` | SSH-терминал через машину (как сейчас) |
| `files` | файловый менеджер на машине |
| `sshfiles` | файловый менеджер на SSH-хосте через машину |
**Разделение кода** (сейчас всё в `terminal.js` и `files.html`):
- `workspace.js` — вкладки, раскладки, разделение, перетаскивание панелей и вкладок, меню «+». С панелями работает через общий интерфейс: `el`, `kind`, `agent`, `title`, `state` и `text` (точка и строка статуса), `fit()`, `focus()`, `close()`, `acceptsDrop(item)`, `drop(item)`;
- `termpane.js` — нынешний класс `Session` (терминал и SSH-терминал), вынесенный почти без изменений;
- `filepane.js` — файловая панель (часть 3);
- `fileviews.js` — общие окна файлов (просмотр и правка, hex, свойства, перемещение, загрузка, редактор SQLite).
**Меню «+»** (на компьютере и телефоне):
- «Терминал ›» — «здесь (машина)», другие машины в сети, «SSH-хост…»;
- «Файловый менеджер ›» — «здесь (машина)», другие машины в сети, «SSH-хост…».
«SSH-хост…» открывает существующее окно SSH-подключения; тип панели зависит от пункта меню. Последний SSH-хост (без секретов) запоминается, как сейчас.
**Из списка машин:** кнопка терминала → новая вкладка-терминал этой машины и переход в «Терминал»; кнопка файлов → новая вкладка «Файлы». Если пространства ещё нет, оно создаётся.
**Заголовки.** Вкладка файлов — иконка папки и `машина · папка`, путь обрезается слева, обновляется при смене папки. Переименование двойным кликом — как у терминалов. Точка статуса файловой панели — машина в сети / офлайн / ошибка.
**Телефон.** Вкладки и меню те же; разделения и перетаскивания, как и сейчас, только на компьютере.
## 3. Файловая панель
Содержимое нынешней страницы файлов как компонент `FilePane(source)`: строка пути, «Вверх», «Обновить», «Папка / Файл / Загрузить», таблица с отметками, «В zip», «Удалить выбранное», загрузка перетаскиванием из ОС.
Состояние (текущая папка, список, отметки) — у экземпляра; элементы ищутся внутри корня панели, без глобальных `id`. Две файловые панели в одной вкладке не мешают друг другу.
**Общие окна** (`fileviews.js`) — одно на страницу. При открытии окну передаётся панель-источник: машина или SSH, путь, что обновить после действия.
**Источник данных** — один интерфейс (`list`, `read`, `write`, `mkdir`, `delete`, `move`, `copy`, `stat`, `chmod`, `download`, `upload`) с двумя реализациями:
- машина — существующий `/api/panel/agents/{id}/fs/...`;
- SSH-хост — новые задачи агента `sshfs.*` (ниже).
**Узкие панели** (меньше ~520 px, в разделённой вкладке): прячется колонка «Изменён», кнопки строки сворачиваются в меню «⋯» — как сейчас на телефоне.
### Файлы по SSH (`sshfiles`)
Агент держит одно SSH-подключение на панель: `sshConnect` и проверка ключа хоста из SSH-терминала. Подключение открывается при создании панели (окно «SSH-хост…»), получает id, закрывается при закрытии панели или через 10 минут простоя. Секреты передаются агенту один раз, панель их не хранит.
Операции — команды на хосте, пути в одинарных кавычках (`shq`):
| операция | команда |
|---|---|
| список | `stat` по содержимому папки; формат выбирается один раз при подключении: GNU/busybox `stat -c`, BSD/macOS `stat -f`; без `stat` — разбор `ls -la` без точного времени |
| чтение, скачивание | `cat -- 'файл'`, поток через stdout |
| запись, загрузка | `cat > 'файл.tmp' && mv 'файл.tmp' 'файл'`, поток через stdin |
| папка, удаление, переименование | `mkdir -p`, `rm -rf`, `mv` |
| права, владелец | `chmod`, `chown` |
| копирование, перенос внутри хоста | `cp -a`, `mv` |
| перенос между хостом и машинами | существующая передача tar по SSH |
Новые задачи агента: `sshfs.open` (подключиться, вернуть id и домашнюю папку), `sshfs.close`, `sshfs.list`, `sshfs.read`, `sshfs.write`, `sshfs.mkdir`, `sshfs.delete`, `sshfs.move`, `sshfs.copy`, `sshfs.stat`, `sshfs.chmod`, `sshfs.send` (скачивание через существующие куски `xfer`), `sshfs.put` (загрузка через куски `xfer`).
Серверу — те же правила доступа, что у файловых операций машины; изменения пишутся в аудит.
Не входит: zip и распаковка на SSH-хосте, редактор SQLite по SSH — эти кнопки в `sshfiles` скрыты. Новых зависимостей нет.
## 4. Перетаскивание
**Что тащится:** строка файла или папки; если строка отмечена — все отмеченные строки панели. У курсора бейдж (имя или «N элементов»).
Механизм — на `pointer`-событиях внутри страницы, тот же, что уже двигает вкладки и панели; подсказка над целью говорит, что произойдёт.
| цель | результат |
|---|---|
| другая файловая панель (любая машина или SSH-хост) | небольшое окно у места броска: «Переместить / Копировать / Отмена», куда (папка цели), «если уже есть: сообщить ▾». Внутри машины — `fs.copy`; внутри SSH-хоста — `cp`/`mv`; между разными — существующая передача tar с прогрессом в этом же окне |
| папка в списке панели | то же, цель — эта папка |
| терминал той же машины / того же SSH-хоста | пути в одинарных кавычках через пробел вставляются в терминал, без Enter |
| терминал другой машины | цель подсвечивается красным, подсказка «Нельзя: файл на X, терминал на Y» |
| заголовок вкладки | через 0,5 с наведения вкладка активируется, бросить можно в её панель |
| пустое место / Esc | отмена |
Правила конфликтов имён — те же, что в окне «Переместить или копировать». Загрузка из ОС перетаскиванием — в любую файловую панель. На телефоне перетаскивания нет; остаётся окно перемещения с выбором машины или SSH.
## 5. Окна машины
«Порты», «Статус», «Изменить» — нынешние страницы как фрагменты (`?frag=1`) в общем окне `machinedlg` (заголовок «машина · раздел», закрытие; на телефоне — на весь экран).
Открываются из кнопок строки в списке машин и из меню заголовка вкладки или панели (её машина). Формы — перехватом из части 1; результат (например, обновления клиента) показывается в окне.
Живые части (графики и опрос статуса, тест скорости, присутствие) — модули `mount` / `unmount`; при закрытии окна останавливаются.
## Ошибки и крайние случаи
- Машина офлайн: терминал — как сейчас; файловая панель — «машина офлайн» и «повторить», при возвращении машины обновляется сама.
- Нет доступа (роль «пользователь»): чужих машин нет в меню и списке; если доступ отобран при открытой вкладке — панель показывает «нет доступа».
- Сессия истекла (401 или редирект на `/login`): сообщение и переход на вход.
- Агент без `sshfs.*`: пункт «Файлы по SSH» сообщает «обновите агента».
- Фрагмент не загрузился: ошибка и «повторить» в его области, остальное приложение работает.
- SSH-подключение файловой панели оборвалось: следующая операция переподключается один раз; не вышло — панель сообщает и предлагает переподключиться.
## Проверка
На стенде (сервер, агент, тестовый sshd), Chrome на компьютере и в эмуляции телефона:
1. Переходы Машины ↔ Терминал ↔ Настройки при идущем выводе в терминале — вывод не прерывается; «Назад / Вперёд»; перезагрузка — сброс и «Покинуть сайт?».
2. Вкладки с терминалом, файлами, SSH-файлами; разделение, изменение размеров, перетаскивание панелей и вкладок.
3. Файловая панель машины и SSH-хоста: список, просмотр, правка, загрузка, скачивание, создание, удаление, переименование, права.
4. Перетаскивание: в соседнюю панель (своя машина, SSH), в папку, в терминал той же машины (вставка путей), в терминал чужой (запрет).
5. Окна машины: формы, обновление клиента, остановка графиков при закрытии.
6. Роли: обычный пользователь видит в меню и списке только свои машины.
7. Регрессия: PIN, ключи доступа, установщик, редактор SQLite, перенос на другие машины, прямой адрес.
## Версия и документация
Версия 0.2.0. README — разделы о рабочем пространстве, файлах по SSH и перетаскивании.
+18 -2
View File
@@ -3,18 +3,34 @@ module github.com/x-grk/longpolling-remote-access
go 1.26
require (
github.com/coder/websocket v1.8.12
github.com/creack/pty v1.1.21
golang.org/x/crypto v0.31.0
github.com/jackc/pgx/v5 v5.7.5
golang.org/x/crypto v0.37.0
golang.org/x/sys v0.32.0
modernc.org/sqlite v1.24.0
)
require (
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
github.com/jackc/puddle/v2 v2.2.2 // indirect
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 // indirect
github.com/mattn/go-isatty v0.0.16 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
golang.org/x/sys v0.28.0 // indirect
golang.org/x/mod v0.17.0 // indirect
golang.org/x/sync v0.13.0 // indirect
golang.org/x/text v0.24.0 // indirect
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d // indirect
lukechampine.com/uint128 v1.2.0 // indirect
modernc.org/cc/v3 v3.40.0 // indirect
modernc.org/ccgo/v3 v3.16.13 // indirect
modernc.org/libc v1.22.5 // indirect
modernc.org/mathutil v1.5.0 // indirect
modernc.org/memory v1.5.0 // indirect
modernc.org/opt v0.1.3 // indirect
modernc.org/strutil v1.1.3 // indirect
modernc.org/token v1.0.1 // indirect
)
+66 -8
View File
@@ -1,28 +1,86 @@
github.com/coder/websocket v1.8.12 h1:5bUXkEPPIbewrnkU8LTCLVaxi4N4J8ahufH2vlo4NAo=
github.com/coder/websocket v1.8.12/go.mod h1:LNVeNrXQZfe5qhS9ALED3uA+l5pPqvwXg3CKoDBB2gs=
github.com/creack/pty v1.1.21 h1:1/QdRyBaHHJP61QkWMXlOIBfsgdDeeKfK8SYVUWJKf0=
github.com/creack/pty v1.1.21/go.mod h1:MOBLtS5ELjhRRrroQr9kyvTxUAFNvYEK993ew/Vr4O4=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/pprof v0.0.0-20221118152302-e6195bd50e26 h1:Xim43kblpZXfIBQsbuBVKCudVG457BR2GZFIz3uw3hQ=
github.com/google/pprof v0.0.0-20221118152302-e6195bd50e26/go.mod h1:dDKJzRmX4S37WGHujM7tX//fmj1uioxKzKxz3lo4HJo=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg=
github.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM=
github.com/jackc/pgpassfile v1.0.0/go.mod h1:CEx0iS5ambNFdcRtxPj5JhEz+xB6uRky5eyVu/W2HEg=
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 h1:iCEnooe7UlwOQYpKFhBabPMi4aNAfoODPEFNiAnClxo=
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761/go.mod h1:5TJZWKEWniPve33vlWYSoGYefn3gLQRzjfDlhSJ9ZKM=
github.com/jackc/pgx/v5 v5.7.5 h1:JHGfMnQY+IEtGM63d+NGMjoRpysB2JBwDr5fsngwmJs=
github.com/jackc/pgx/v5 v5.7.5/go.mod h1:aruU7o91Tc2q2cFp5h4uP3f6ztExVpyVv88Xl/8Vl8M=
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 h1:Z9n2FFNUXsshfwJMBgNA0RU6/i7WVaAegv3PtuIHPMs=
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51/go.mod h1:CzGEWj7cYgsdH8dAjBGEr58BoE7ScuLd+fwFZ44+/x8=
github.com/mattn/go-isatty v0.0.16 h1:bq3VjFmv/sOjHtdEhmkEV4x1AJtvUvOJ2PFAZ5+peKQ=
github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
github.com/mattn/go-sqlite3 v1.14.16 h1:yOQRA0RpS5PFz/oikGwBEqvAWhWg5ufRz4ETLjwpU1Y=
github.com/mattn/go-sqlite3 v1.14.16/go.mod h1:2eHXhiwb8IkHr+BDWZGa96P6+rkvnG63S2DGjv9HUNg=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
golang.org/x/crypto v0.28.0 h1:GBDwsMXVQi34v5CCYUm2jkJvu4cbtru2U4TN2PSyQnw=
golang.org/x/crypto v0.28.0/go.mod h1:rmgy+3RHxRZMyY0jjAJShp2zgEdOqj2AO7U0pYmeQ7U=
golang.org/x/crypto v0.31.0 h1:ihbySMvVjLAeSH1IbfcRTkD/iNscyz8rGzjF/E5hV6U=
golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.1 h1:w7B6lhMri9wdJUVmEZPGGhZzrYTPvgJArz7wNPgYKsk=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
golang.org/x/crypto v0.37.0 h1:kJNSjF/Xp7kU0iB2Z+9viTPMW4EqqsrywMXLJOOsXSE=
golang.org/x/crypto v0.37.0/go.mod h1:vg+k43peMZ0pUMhYmVAWysMK35e6ioLh3wB8ZCAfbVc=
golang.org/x/mod v0.17.0 h1:zY54UmvipHiNd+pm+m0x9KhZ9hl1/7QNMyxXbc6ICqA=
golang.org/x/mod v0.17.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/sync v0.13.0 h1:AauUjRAJ9OSnvULf/ARrrVywoJDy0YS2AwQ98I37610=
golang.org/x/sync v0.13.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.28.0 h1:Fksou7UEQUWlKvIdsqzJmUmCX3cZuD2+P3XyyzwMhlA=
golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.32.0 h1:s77OFDvIQeibCmezSnk/q6iAfkdiQaJi4VzroCFrN20=
golang.org/x/sys v0.32.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/term v0.31.0 h1:erwDkOK1Msy6offm1mOgvspSkslFnIGsFnxOKoufg3o=
golang.org/x/term v0.31.0/go.mod h1:R4BeIy7D95HzImkxGkTW1UQTtP54tio2RyHz7PwK0aw=
golang.org/x/text v0.24.0 h1:dd5Bzh4yt5KYA8f9CJHCP4FB4D51c2c6JvN37xJJkJ0=
golang.org/x/text v0.24.0/go.mod h1:L8rBsPeo2pSS+xqN0d5u2ikmjtmoJbDBT1b7nHvFCdU=
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d h1:vU5i/LfpvrRCpgM/VPfJLg5KjxD3E+hfT1SH+d9zLwg=
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d/go.mod h1:aiJjzUbINMkxbQROHiO6hDPo2LHcIPhhQsa9DLh0yGk=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
lukechampine.com/uint128 v1.2.0 h1:mBi/5l91vocEN8otkC5bDLhi2KdCticRiwbdB0O+rjI=
lukechampine.com/uint128 v1.2.0/go.mod h1:c4eWIwlEGaxC/+H1VguhU4PHXNWDCDMUlWdIWl2j1gk=
modernc.org/cc/v3 v3.40.0 h1:P3g79IUS/93SYhtoeaHW+kRCIrYaxJ27MFPv+7kaTOw=
modernc.org/cc/v3 v3.40.0/go.mod h1:/bTg4dnWkSXowUO6ssQKnOV0yMVxDYNIsIrzqTFDGH0=
modernc.org/ccgo/v3 v3.16.13 h1:Mkgdzl46i5F/CNR/Kj80Ri59hC8TKAhZrYSaqvkwzUw=
modernc.org/ccgo/v3 v3.16.13/go.mod h1:2Quk+5YgpImhPjv2Qsob1DnZ/4som1lJTodubIcoUkY=
modernc.org/ccorpus v1.11.6 h1:J16RXiiqiCgua6+ZvQot4yUuUy8zxgqbqEEUuGPlISk=
modernc.org/ccorpus v1.11.6/go.mod h1:2gEUTrWqdpH2pXsmTM1ZkjeSrUWDpjMu2T6m29L/ErQ=
modernc.org/httpfs v1.0.6 h1:AAgIpFZRXuYnkjftxTAZwMIiwEqAfk8aVB2/oA6nAeM=
modernc.org/httpfs v1.0.6/go.mod h1:7dosgurJGp0sPaRanU53W4xZYKh14wfzX420oZADeHM=
modernc.org/libc v1.22.5 h1:91BNch/e5B0uPbJFgqbxXuOnxBQjlS//icfQEGmvyjE=
modernc.org/libc v1.22.5/go.mod h1:jj+Z7dTNX8fBScMVNRAYZ/jF91K8fdT2hYMThc3YjBY=
modernc.org/mathutil v1.5.0 h1:rV0Ko/6SfM+8G+yKiyI830l3Wuz1zRutdslNoQ0kfiQ=
modernc.org/mathutil v1.5.0/go.mod h1:mZW8CKdRPY1v87qxC/wUdX5O1qDzXMP5TH3wjfpga6E=
modernc.org/memory v1.5.0 h1:N+/8c5rE6EqugZwHii4IFsaJ7MUhoWX07J5tC/iI5Ds=
modernc.org/memory v1.5.0/go.mod h1:PkUhL0Mugw21sHPeskwZW4D6VscE/GQJOnIpCnW6pSU=
modernc.org/opt v0.1.3 h1:3XOZf2yznlhC+ibLltsDGzABUGVx8J6pnFMS3E4dcq4=
modernc.org/opt v0.1.3/go.mod h1:WdSiB5evDcignE70guQKxYUl14mgWtbClRi5wmkkTX0=
modernc.org/sqlite v1.24.0 h1:EsClRIWHGhLTCX44p+Ri/JLD+vFGo0QGjasg2/F9TlI=
modernc.org/sqlite v1.24.0/go.mod h1:OrDj17Mggn6MhE+iPbBNf7RGKODDE9NFT0f3EwDzJqk=
modernc.org/strutil v1.1.3 h1:fNMm+oJklMGYfU9Ylcywl0CO5O6nTfaowNsh2wpPjzY=
modernc.org/strutil v1.1.3/go.mod h1:MEHNA7PdEnEwLvspRMtWTNnp2nnyvMfkimT1NKNAGbw=
modernc.org/tcl v1.15.2 h1:C4ybAYCGJw968e+Me18oW55kD/FexcHbqH2xak1ROSY=
modernc.org/tcl v1.15.2/go.mod h1:3+k/ZaEbKrC8ePv8zJWPtBSW0V7Gg9g8rkmhI1Kfs3c=
modernc.org/token v1.0.1 h1:A3qvTqOwexpfZZeyI0FeGPDlSWX5pjZu9hF4lU+EKWg=
modernc.org/token v1.0.1/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=
modernc.org/z v1.7.3 h1:zDJf6iHjrnB+WRD88stbXokugjyc0/pB91ri1gO6LZY=
modernc.org/z v1.7.3/go.mod h1:Ipv4tsdxZRbQyLq9Q1M6gdbkxYzdlrciF2Hi/lS7nWE=
+250 -21
View File
@@ -13,38 +13,57 @@ import (
"log"
"net"
"net/http"
"net/url"
"os"
"path/filepath"
"runtime"
"sync"
"sync/atomic"
"time"
lpra "github.com/x-grk/longpolling-remote-access"
"github.com/x-grk/longpolling-remote-access/internal/proto"
)
// Version is the agent build version, injected at compile time via -ldflags
// (-X .../internal/agent.Version=...).
var Version = "dev"
// Version is the agent build version (see VERSION.txt).
var Version = lpra.Version
// Config is the agent's runtime configuration (from env / config.env).
type Config struct {
Host string // e.g. https://REDACTED:8445
Fallback string // the direct address, from the installer (FALLBACK_HOST); may be empty
EnrollSecret string
Name string // optional friendly name
StatePath string // where the enrolled id+token are persisted
TunnelAddr string // host:port for the direct forward tunnel (e.g. REDACTED:8446); empty disables
TunnelAddr string // host:port for the direct forward tunnel (e.g. panel.example:port); empty disables
// TunnelPinned keeps TunnelAddr as configured locally (TUNNEL_ADDR /
// TUNNEL_PORT) instead of following the address set in the panel.
TunnelPinned bool
}
type state struct {
AgentID string `json:"agent_id"`
Token string `json:"token"`
// Fallback is the direct address as the server told it.
Fallback string `json:"fallback,omitempty"`
}
// Agent is a running client.
type Agent struct {
cfg Config
http *http.Client
stream *http.Client
state state
stateMu sync.Mutex
hosts hostState
pollCount int
// Direct tunnel currently running (see setTunnelAddr).
transport atomic.Value // string: auto, ws or lp, as the server says
link atomic.Pointer[wsLink]
tunMu sync.Mutex
tunAddr string
tunCancel context.CancelFunc
}
// New builds an agent. It does not perform any network I/O.
@@ -71,6 +90,8 @@ func New(cfg Config) *Agent {
cfg: cfg,
// Timeout must exceed the server's long-poll hold (25s) plus slack.
http: &http.Client{Timeout: 60 * time.Second, Transport: tr},
// File streams take as long as the file needs.
stream: &http.Client{Transport: tr},
}
}
@@ -81,37 +102,164 @@ func (a *Agent) Run(ctx context.Context) error {
return err
}
if a.state.AgentID == "" || a.state.Token == "" {
if err := a.enroll(ctx); err != nil {
err := a.enroll(ctx)
// The main address out of reach at install: the direct one.
if err != nil && a.cfg.Fallback != "" && a.cfg.Fallback != a.cfg.Host {
log.Printf("enroll at %s: %v - trying %s", a.cfg.Host, err, a.cfg.Fallback)
a.setHost(a.cfg.Fallback)
a.hostChanged()
err = a.enroll(ctx)
}
if err != nil {
return fmt.Errorf("enroll: %w", err)
}
}
log.Printf("agent %s connected to %s", a.state.AgentID, a.cfg.Host)
go a.watchMain(ctx)
log.Printf("agent %s connected to %s", a.state.AgentID, a.host())
// Direct forward tunnel (low-latency TCP mux + UDP) on 8446, if configured.
if a.cfg.TunnelAddr != "" {
go a.runTunnel(ctx)
go a.runUDP(ctx)
// Direct forward tunnel (low-latency TCP mux + UDP). Its address comes
// from the panel; fall back to the configured one when the server sends
// none (tunnel port left at the default on an older server).
if a.cfg.TunnelPinned || !a.fetchConfig(ctx) {
a.setTunnelAddr(ctx, a.cfg.TunnelAddr)
}
// The control channel: a WebSocket, or long polls. In "auto" a failed or
// dropped WebSocket gives way to long polling for a minute, then is tried
// again; "ws" keeps trying the WebSocket; "lp" only polls.
backoff := time.Second
for ctx.Err() == nil {
if err := a.pollOnce(ctx); err != nil {
log.Printf("poll: %v (retry in %s)", err, backoff)
if a.mode() == proto.TransportLP {
a.longPoll(ctx, 0)
continue
}
err := a.runWS(ctx)
if ctx.Err() != nil {
return nil
}
// Moved to the other address: the WebSocket goes there at once.
if a.hostChanged() {
backoff = time.Second
continue
}
a.failed()
if a.hostChanged() {
continue
}
switch a.mode() {
case proto.TransportWS:
log.Printf("websocket: %v (retry in %s)", err, backoff)
select {
case <-time.After(backoff):
case <-ctx.Done():
return nil
}
if backoff < 15*time.Second {
backoff *= 2
}
backoff = min(backoff*2, retryCap)
continue
case proto.TransportAuto:
log.Printf("websocket: %v - long polling, websocket again in a minute", err)
}
backoff = time.Second
a.longPoll(ctx, time.Minute)
}
return nil
}
// mode is the transport the server set for this agent.
func (a *Agent) mode() string {
if v, ok := a.transport.Load().(string); ok {
return v
}
return proto.TransportAuto
}
func (a *Agent) applyTransport(v string) {
switch v {
case proto.TransportAuto, proto.TransportWS, proto.TransportLP:
if v != a.mode() {
log.Printf("transport: %s", v)
a.transport.Store(v)
}
}
}
// longPoll runs control polls: for d, or with d zero until the transport is
// no longer long polling.
func (a *Agent) longPoll(ctx context.Context, d time.Duration) {
until := time.Now().Add(d)
backoff := time.Second
// The server was out of reach (a network down, a machine asleep): once
// a poll gets through again, "auto" tries the WebSocket at once rather
// than at the end of the minute - it was the network, not the WebSocket.
lost := false
for ctx.Err() == nil {
if d > 0 && (time.Now().After(until) || a.mode() == proto.TransportWS) {
return
}
if d == 0 && a.mode() != proto.TransportLP {
return
}
// Back from out of reach in "auto": a quick request tells when the
// server answers again, and the WebSocket is tried then; a poll
// would sit held on the server first.
if lost && d > 0 {
if a.reachable(ctx) {
a.reached()
return
}
a.failed()
if a.hostChanged() {
return
}
select {
case <-time.After(backoff):
case <-ctx.Done():
return
}
backoff = min(backoff*2, retryCap)
continue
}
// In "auto" the minute is a minute: a poll still held then is cut
// short (the server keeps its job for the next one).
reqCtx, cancel := ctx, context.CancelFunc(func() {})
if d > 0 {
reqCtx, cancel = context.WithDeadline(ctx, until)
}
err := a.pollOnce(ctx, reqCtx)
cut := reqCtx.Err() == context.DeadlineExceeded
cancel()
if err != nil && cut && ctx.Err() == nil {
return
}
if err != nil {
a.failed()
if a.hostChanged() {
return
}
log.Printf("poll: %v (retry in %s)", err, backoff)
select {
case <-time.After(backoff):
case <-ctx.Done():
return
}
backoff = min(backoff*2, retryCap)
lost = true
continue
}
a.reached()
backoff = time.Second
}
}
// reachable reports whether the server answers a quick request.
func (a *Agent) reachable(ctx context.Context) bool {
return a.reachableAt(ctx, a.host())
}
// retryCap bounds the wait between attempts to reach the server, so a
// machine back from sleep or a network back up is reconnected in seconds.
const retryCap = 5 * time.Second
func (a *Agent) enroll(ctx context.Context) error {
host, _ := os.Hostname()
body, _ := json.Marshal(proto.EnrollRequest{
@@ -122,7 +270,7 @@ func (a *Agent) enroll(ctx context.Context) error {
Version: Version,
User: currentUser(),
})
req, _ := http.NewRequestWithContext(ctx, http.MethodPost, a.cfg.Host+"/api/agent/enroll", bytes.NewReader(body))
req, _ := http.NewRequestWithContext(ctx, http.MethodPost, a.host()+"/api/agent/enroll", bytes.NewReader(body))
req.Header.Set("X-Enroll-Secret", a.cfg.EnrollSecret)
req.Header.Set("Content-Type", "application/json")
resp, err := a.http.Do(req)
@@ -143,8 +291,10 @@ func (a *Agent) enroll(ctx context.Context) error {
}
// pollOnce performs one control long-poll and dispatches any instruction.
func (a *Agent) pollOnce(ctx context.Context) error {
req, _ := http.NewRequestWithContext(ctx, http.MethodGet, a.cfg.Host+"/api/agent/poll", nil)
// The poll itself runs on reqCtx, which may end early; what it starts runs on
// ctx.
func (a *Agent) pollOnce(ctx, reqCtx context.Context) error {
req, _ := http.NewRequestWithContext(reqCtx, http.MethodGet, a.host()+"/api/agent/poll", nil)
a.authHeaders(req)
// Report the version and our OS user occasionally (not every poll), so the
// panel can flag outdated agents without extra traffic.
@@ -168,16 +318,93 @@ func (a *Agent) pollOnce(ctx context.Context) error {
if err := json.NewDecoder(resp.Body).Decode(&msg); err != nil {
return err
}
// Dispatch in the background so we keep polling for more work.
a.applyTunnelHeader(ctx, resp.Header)
a.learnAddrs(resp.Header.Get(proto.MainHeader), resp.Header.Get(proto.FallbackHeader))
a.applyTransport(resp.Header.Get(proto.TransportHeader))
a.dispatch(ctx, msg)
return nil
}
// dispatch starts what a control message asks for, in the background, so the
// control channel keeps listening.
func (a *Agent) dispatch(ctx context.Context, msg proto.ControlMessage) {
switch msg.Type {
case proto.CtrlOpenTerminal:
go a.runTerminal(ctx, msg.SessionID, msg.User)
go a.runTerminal(ctx, msg.SessionID, msg.User, msg.Dir)
case proto.CtrlOpenForward:
go a.runForward(ctx, msg.SessionID, msg.Port)
case proto.CtrlTask:
go a.runTask(ctx, msg.TaskID)
case proto.CtrlOpenSSH:
if msg.SSH != nil {
go a.runSSH(ctx, msg.SessionID, *msg.SSH)
}
return nil
}
}
// fetchConfig asks the server for the settings it hands to agents and reports
// whether it named a tunnel address.
func (a *Agent) fetchConfig(ctx context.Context) bool {
ctx2, cancel := context.WithTimeout(ctx, 10*time.Second)
defer cancel()
req, _ := http.NewRequestWithContext(ctx2, http.MethodGet, a.host()+"/api/agent/config", nil)
a.authHeaders(req)
resp, err := a.http.Do(req)
if err != nil {
return false
}
defer resp.Body.Close()
a.applyTransport(resp.Header.Get(proto.TransportHeader))
a.learnAddrs(resp.Header.Get(proto.MainHeader), resp.Header.Get(proto.FallbackHeader))
return a.applyTunnelHeader(ctx, resp.Header)
}
// applyTunnelHeader follows the tunnel address the server advertises as
// "host:port"; an empty host means the host we already reach the server on.
func (a *Agent) applyTunnelHeader(ctx context.Context, h http.Header) bool {
return a.applyTunnelAddr(ctx, h.Get("X-Tunnel-Addr"))
}
func (a *Agent) applyTunnelAddr(ctx context.Context, v string) bool {
if v == "" || a.cfg.TunnelPinned {
return false
}
host, port, err := net.SplitHostPort(v)
if err != nil {
return false
}
if host == "" {
u, err := url.Parse(a.host())
if err != nil || u.Hostname() == "" {
return false
}
host = u.Hostname()
}
a.setTunnelAddr(ctx, net.JoinHostPort(host, port))
return true
}
// setTunnelAddr points the direct tunnel at addr, restarting it if it was
// running against another address. An empty addr stops it.
func (a *Agent) setTunnelAddr(ctx context.Context, addr string) {
a.tunMu.Lock()
defer a.tunMu.Unlock()
if addr == a.tunAddr {
return
}
if a.tunCancel != nil {
a.tunCancel()
a.tunCancel = nil
}
a.tunAddr = addr
if addr == "" {
return
}
log.Printf("tunnel: using %s", addr)
tctx, cancel := context.WithCancel(ctx)
a.tunCancel = cancel
go a.runTunnel(tctx, addr)
go a.runUDP(tctx, addr)
}
func (a *Agent) authHeaders(req *http.Request) {
@@ -202,7 +429,9 @@ func (a *Agent) saveState() error {
if err := os.MkdirAll(filepath.Dir(a.cfg.StatePath), 0o700); err != nil {
return err
}
a.stateMu.Lock()
b, _ := json.MarshalIndent(a.state, "", " ")
a.stateMu.Unlock()
return os.WriteFile(a.cfg.StatePath, b, 0o600)
}
+63 -4
View File
@@ -9,7 +9,9 @@ import (
"io"
"os"
"os/exec"
"os/user"
"path/filepath"
"runtime"
"sort"
"strings"
"unicode/utf8"
@@ -17,8 +19,8 @@ import (
"github.com/x-grk/longpolling-remote-access/internal/proto"
)
// fsReadCap bounds how many bytes a view/read returns (5 MiB).
const fsReadCap = 5 * 1024 * 1024
// fsReadCap bounds how many bytes a view/read returns.
const fsReadCap = proto.FSReadCap
// fsDownloadCap bounds a browser download (kept under the 64 MiB result/body
// limit once base64-encoded).
@@ -70,6 +72,18 @@ func dispatchFS(env proto.TaskEnvelope) (any, error) {
var a proto.FSZipArgs
_ = json.Unmarshal(env.Args, &a)
return fsZip(a.Paths, a.Dest)
case proto.TaskFSStat:
var a proto.FSPathArgs
_ = json.Unmarshal(env.Args, &a)
return fsStat(a.Path)
case proto.TaskFSChmod:
var a proto.FSChmodArgs
_ = json.Unmarshal(env.Args, &a)
return fsChmod(a)
case proto.TaskFSCopy:
var a proto.FSCopyArgs
_ = json.Unmarshal(env.Args, &a)
return fsCopy(a)
default:
return nil, fmt.Errorf("неизвестная операция %q", env.Type)
}
@@ -79,8 +93,18 @@ func fsList(path string) (proto.FSListResult, error) {
if path == "" {
path = "/"
}
// "~" / "~name" is where the file manager starts: the home directory, or
// the root when there is none to open.
home := strings.HasPrefix(path, "~") && !strings.Contains(path, "/")
if home {
path = homeDir(path[1:])
}
abs, _ := filepath.Abs(path)
entries, err := os.ReadDir(abs)
if err != nil && home && abs != "/" {
abs = "/"
entries, err = os.ReadDir(abs)
}
if err != nil {
return proto.FSListResult{}, err
}
@@ -90,13 +114,21 @@ func fsList(path string) (proto.FSListResult, error) {
if err != nil {
continue
}
out = append(out, proto.FSEntry{
ent := proto.FSEntry{
Name: e.Name(),
IsDir: e.IsDir(),
Size: info.Size(),
Mode: info.Mode().String(),
MTime: info.ModTime().Unix(),
})
}
// A link to a folder opens like one.
if info.Mode()&os.ModeSymlink != 0 {
ent.Link = true
if t, err := os.Stat(filepath.Join(abs, e.Name())); err == nil && t.IsDir() {
ent.IsDir = true
}
}
out = append(out, ent)
}
// Directories first, then alphabetical.
sort.Slice(out, func(i, j int) bool {
@@ -222,3 +254,30 @@ func fsZip(paths []string, dest string) (any, error) {
}
return map[string]string{"dest": dest}, nil
}
// homeDir returns the home directory of the named user ("" = the user the
// agent runs as), or "/" when it cannot be told.
func homeDir(name string) string {
if name != "" {
if validUserName(name) {
if u, err := user.Lookup(name); err == nil && u.HomeDir != "" {
return u.HomeDir
}
}
return "/"
}
if u, err := user.Current(); err == nil && u.HomeDir != "" {
return u.HomeDir
}
if h := os.Getenv("HOME"); h != "" {
return h
}
// A root service often has neither a passwd entry we can read nor $HOME.
if os.Geteuid() == 0 {
if runtime.GOOS == "darwin" {
return "/var/root"
}
return "/root"
}
return "/"
}
+166
View File
@@ -0,0 +1,166 @@
package agent
import (
"errors"
"fmt"
"io"
"io/fs"
"os"
"path/filepath"
"strings"
"syscall"
"github.com/x-grk/longpolling-remote-access/internal/proto"
)
// fsCopy copies or moves Src to Dst, following Conflict when Dst exists.
func fsCopy(a proto.FSCopyArgs) (any, error) {
src, dst := filepath.Clean(a.Src), filepath.Clean(a.Dst)
if a.Src == "" || a.Dst == "" {
return nil, errors.New("не указан путь")
}
if src == dst {
return nil, errors.New("откуда и куда - одно и то же")
}
if strings.HasPrefix(dst, src+string(filepath.Separator)) {
return nil, errors.New("нельзя положить папку внутрь неё самой")
}
if _, err := os.Lstat(src); err != nil {
return nil, err
}
if _, err := os.Stat(filepath.Dir(dst)); err != nil {
return nil, fmt.Errorf("нет папки %s", filepath.Dir(dst))
}
c := &copier{move: a.Move, conflict: a.Conflict}
if err := c.place(src, dst); err != nil {
return map[string]int{"done": c.done, "skipped": c.skipped}, err
}
return map[string]int{"done": c.done, "skipped": c.skipped}, nil
}
type copier struct {
move bool
conflict string
done, skipped int
}
// place puts src at dst. Two folders merge when the rule allows it; anything
// else that is in the way stops, is replaced or is kept, by the rule.
func (c *copier) place(src, dst string) error {
si, err := os.Lstat(src)
if err != nil {
return err
}
di, err := os.Lstat(dst)
if err == nil {
if c.conflict != "replace" && c.conflict != "skip" {
return fmt.Errorf("%s уже существует", dst)
}
if si.IsDir() && di.IsDir() {
entries, err := os.ReadDir(src)
if err != nil {
return err
}
for _, e := range entries {
if err := c.place(filepath.Join(src, e.Name()), filepath.Join(dst, e.Name())); err != nil {
return err
}
}
if c.move {
// Whatever was skipped stays where it was.
_ = os.Remove(src)
}
return nil
}
if c.conflict == "skip" {
c.skipped++
return nil
}
if err := os.RemoveAll(dst); err != nil {
return err
}
} else if !errors.Is(err, fs.ErrNotExist) {
return err
}
if c.move {
err := os.Rename(src, dst)
if err == nil {
c.done++
return nil
}
// Another file system: copy, then take the original away.
if !errors.Is(err, syscall.EXDEV) {
return err
}
if err := c.copyTree(src, dst); err != nil {
return err
}
return os.RemoveAll(src)
}
return c.copyTree(src, dst)
}
// copyTree copies a file, a symlink or a whole folder, keeping modes and
// times.
func (c *copier) copyTree(src, dst string) error {
return filepath.WalkDir(src, func(p string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
rel, _ := filepath.Rel(src, p)
to := filepath.Join(dst, rel)
info, err := os.Lstat(p)
if err != nil {
return err
}
switch {
case info.Mode()&fs.ModeSymlink != 0:
link, err := os.Readlink(p)
if err != nil {
return err
}
if err := os.Symlink(link, to); err != nil {
return err
}
case info.IsDir():
if err := os.Mkdir(to, info.Mode().Perm()); err != nil && !errors.Is(err, fs.ErrExist) {
return err
}
case info.Mode().IsRegular():
if err := copyFile(p, to, info); err != nil {
return err
}
default:
return nil // sockets, devices: not copied
}
c.done++
if st, ok := info.Sys().(*syscall.Stat_t); ok {
_ = os.Lchown(to, int(st.Uid), int(st.Gid))
}
if info.Mode()&fs.ModeSymlink == 0 {
_ = os.Chtimes(to, info.ModTime(), info.ModTime())
}
return nil
})
}
func copyFile(src, dst string, info fs.FileInfo) error {
in, err := os.Open(src)
if err != nil {
return err
}
defer in.Close()
out, err := os.OpenFile(dst, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, info.Mode().Perm())
if err != nil {
return err
}
if _, err := io.Copy(out, in); err != nil {
out.Close()
os.Remove(dst)
return err
}
if err := out.Close(); err != nil {
return err
}
return os.Chmod(dst, info.Mode()&(fs.ModePerm|fs.ModeSetuid|fs.ModeSetgid|fs.ModeSticky))
}
+163
View File
@@ -0,0 +1,163 @@
package agent
import (
"bufio"
"errors"
"fmt"
"io/fs"
"os"
"os/user"
"path/filepath"
"sort"
"strconv"
"strings"
"syscall"
"github.com/x-grk/longpolling-remote-access/internal/proto"
)
// fsStat reports a file's properties, following a symlink to what it points
// at, and the user and group names to choose an owner from.
func fsStat(path string) (proto.FSStatResult, error) {
res := proto.FSStatResult{Path: path}
if li, err := os.Lstat(path); err == nil && li.Mode()&os.ModeSymlink != 0 {
res.Link, _ = os.Readlink(path)
}
info, err := os.Stat(path)
if err != nil {
return res, err
}
res.IsDir, res.Size, res.MTime = info.IsDir(), info.Size(), info.ModTime().Unix()
if st, ok := info.Sys().(*syscall.Stat_t); ok {
res.Mode = uint32(st.Mode) & 07777
res.UID, res.GID = int(st.Uid), int(st.Gid)
} else {
res.Mode = uint32(info.Mode().Perm())
}
res.Owner, res.Group = strconv.Itoa(res.UID), strconv.Itoa(res.GID)
if u, err := user.LookupId(res.Owner); err == nil {
res.Owner = u.Username
}
if g, err := user.LookupGroupId(res.Group); err == nil {
res.Group = g.Name
}
res.Users, res.Groups = namesIn("/etc/passwd"), namesIn("/etc/group")
return res, nil
}
// namesIn lists the first field of a passwd- or group-style file.
func namesIn(file string) []string {
f, err := os.Open(file)
if err != nil {
return nil
}
defer f.Close()
var out []string
sc := bufio.NewScanner(f)
for sc.Scan() && len(out) < 1000 {
line := sc.Text()
if line == "" || line[0] == '#' {
continue
}
if name, _, ok := strings.Cut(line, ":"); ok && name != "" {
out = append(out, name)
}
}
sort.Strings(out)
return out
}
// lookupID resolves a user or group given by name or number; -1 for none.
func lookupID(name string, group bool) (int, error) {
if name == "" {
return -1, nil
}
if n, err := strconv.Atoi(name); err == nil && n >= 0 {
return n, nil
}
if group {
g, err := user.LookupGroup(name)
if err != nil {
return -1, fmt.Errorf("группа %s не найдена", name)
}
return strconv.Atoi(g.Gid)
}
u, err := user.Lookup(name)
if err != nil {
return -1, fmt.Errorf("пользователь %s не найден", name)
}
return strconv.Atoi(u.Uid)
}
// fsChmod sets mode, owner and group on a path, and with Recursive on all
// that is inside a folder. Symlinks inside are not followed.
func fsChmod(a proto.FSChmodArgs) (any, error) {
if a.Path == "" {
return nil, errors.New("не указан путь")
}
if a.Recursive && filepath.Clean(a.Path) == "/" {
return nil, errors.New("отказ: рекурсивно для корня нельзя")
}
if a.Mode != nil && *a.Mode > 07777 {
return nil, errors.New("неверные права")
}
uid, err := lookupID(a.Owner, false)
if err != nil {
return nil, err
}
gid, err := lookupID(a.Group, true)
if err != nil {
return nil, err
}
changed, failed := 0, 0
var errs []string
// The path itself always gets the change; Only picks among what is in it.
apply := func(p string, isDir, isLink, self bool) {
if !self && ((a.Only == "files" && isDir) || (a.Only == "dirs" && !isDir)) {
return
}
var err error
if uid >= 0 || gid >= 0 {
err = os.Lchown(p, uid, gid)
}
// A symlink has no mode of its own: chmod would change its target.
if err == nil && a.Mode != nil && !isLink {
err = syscall.Chmod(p, *a.Mode)
}
if err != nil {
failed++
if len(errs) < 5 {
errs = append(errs, err.Error())
}
return
}
changed++
}
info, err := os.Stat(a.Path)
if err != nil {
return nil, err
}
if !a.Recursive || !info.IsDir() {
apply(a.Path, info.IsDir(), false, true)
} else {
err = filepath.WalkDir(a.Path, func(p string, d fs.DirEntry, err error) error {
if err != nil {
failed++
if len(errs) < 5 {
errs = append(errs, err.Error())
}
return nil
}
apply(p, d.IsDir(), d.Type()&fs.ModeSymlink != 0, p == a.Path)
return nil
})
if err != nil {
return nil, err
}
}
res := map[string]any{"changed": changed, "failed": failed, "errors": errs}
if failed > 0 && changed == 0 {
return res, errors.New(errs[0])
}
return res, nil
}
+337
View File
@@ -0,0 +1,337 @@
package agent
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"os"
"path/filepath"
"strconv"
"strings"
"sync"
"syscall"
"time"
"github.com/x-grk/longpolling-remote-access/internal/proto"
)
// Files go to and from the browser in pieces, each a request of its own
// through the server: small enough for any proxy in front, and sent again
// alone when the link drops it. A few go at once, so a good link is kept as
// busy as by one long stream.
const (
// xferWorkers is how many pieces are in flight at once.
xferWorkers = 4
// xferPieceTimeout bounds one piece's request: a link that slow, or
// stuck, gets the piece sent again.
xferPieceTimeout = 5 * time.Minute
// xferTries bounds how often a piece is tried over a failing link.
xferTries = 30
)
// errXferGone is the server saying the transfer is over: the browser left
// or gave up.
type errXferGone string
func (e errXferGone) Error() string { return string(e) }
func (a *Agent) xferURL(id string, i int) string {
return a.host() + "/api/agent/fs/xfer/" + id + "/" + strconv.Itoa(i)
}
// xferRequest makes one request for a piece, trying again over a failing link.
// A 503 is the server waiting on the other side: asked again at once.
func (a *Agent) xferRequest(ctx context.Context, newReq func(context.Context) (*http.Request, error), keep func(*http.Response) error) error {
fails := 0
for {
if err := ctx.Err(); err != nil {
return err
}
rctx, cancel := context.WithTimeout(ctx, xferPieceTimeout)
req, err := newReq(rctx)
if err != nil {
cancel()
return err
}
a.authHeaders(req)
resp, err := a.stream.Do(req)
if err == nil {
switch {
case resp.StatusCode == http.StatusGone:
msg, _ := io.ReadAll(io.LimitReader(resp.Body, 512))
resp.Body.Close()
cancel()
return errXferGone(strings.TrimSpace(string(msg)))
case resp.StatusCode == http.StatusServiceUnavailable:
io.Copy(io.Discard, resp.Body)
resp.Body.Close()
cancel()
continue
case resp.StatusCode < 300:
err = keep(resp)
resp.Body.Close()
cancel()
if err == nil {
return nil
}
default:
io.Copy(io.Discard, resp.Body)
resp.Body.Close()
err = errStatus(resp.StatusCode)
}
}
cancel()
if fails++; fails >= xferTries {
return fmt.Errorf("сеть не держит передачу: %w", err)
}
select {
case <-time.After(min(time.Duration(fails)*time.Second, 10*time.Second)):
case <-ctx.Done():
return ctx.Err()
}
}
}
// pieces runs fn for every piece, xferWorkers at a time, until one fails.
func pieces(ctx context.Context, n int, fn func(ctx context.Context, i int) error) error {
ctx, cancel := context.WithCancel(ctx)
defer cancel()
var (
mu sync.Mutex
next int
first error
wg sync.WaitGroup
)
for w := 0; w < min(xferWorkers, n); w++ {
wg.Add(1)
go func() {
defer wg.Done()
for {
mu.Lock()
if first != nil || next >= n {
mu.Unlock()
return
}
i := next
next++
mu.Unlock()
if err := fn(ctx, i); err != nil {
mu.Lock()
if first == nil {
first = err
cancel()
}
mu.Unlock()
return
}
}
}()
}
wg.Wait()
return first
}
func pieceCount(size int64, chunk int) int {
if size <= 0 {
return 1
}
return int((size + int64(chunk) - 1) / int64(chunk))
}
// doFSPut writes the file the browser is uploading. It lands in a temporary
// file next to the target first, so a broken upload leaves the old file as it
// was.
func (a *Agent) doFSPut(ctx context.Context, env proto.TaskEnvelope) proto.TaskResult {
var args proto.FSXferArgs
if err := json.Unmarshal(env.Args, &args); err != nil {
return proto.TaskResult{Error: err.Error()}
}
if args.Size < 0 || args.Size > proto.FSFileCap || args.Chunk <= 0 {
return proto.TaskResult{Error: "файл больше 512 МБ"}
}
dir, base := filepath.Split(args.Path)
if dir == "" {
dir = "."
}
if info, err := os.Stat(args.Path); err == nil && info.IsDir() {
return proto.TaskResult{Error: "это каталог"}
}
tmp, err := os.CreateTemp(dir, "."+base+".lpra-*")
if err != nil {
return proto.TaskResult{Error: err.Error()}
}
done := false
defer func() {
if !done {
tmp.Close()
os.Remove(tmp.Name())
}
}()
// Pieces written so far, to tell the server which it can let go.
n := pieceCount(args.Size, args.Chunk)
var (
mu sync.Mutex
got = make([]bool, n)
contig int
)
err = pieces(ctx, n, func(ctx context.Context, i int) error {
want := min(int64(args.Chunk), args.Size-int64(i)*int64(args.Chunk))
mu.Lock()
ack := contig
mu.Unlock()
var data []byte
err := a.xferRequest(ctx, func(ctx context.Context) (*http.Request, error) {
return http.NewRequestWithContext(ctx, http.MethodGet, a.xferURL(args.Xfer, i)+"?ack="+strconv.Itoa(ack), nil)
}, func(resp *http.Response) error {
b, err := io.ReadAll(io.LimitReader(resp.Body, int64(args.Chunk)+1))
if err == nil && int64(len(b)) != want {
err = errors.New("кусок не той длины")
}
data = b
return err
})
if err != nil {
return err
}
if _, err := tmp.WriteAt(data, int64(i)*int64(args.Chunk)); err != nil {
return err
}
mu.Lock()
got[i] = true
for contig < n && got[contig] {
contig++
}
mu.Unlock()
return nil
})
if err != nil {
var gone errXferGone
if errors.As(err, &gone) {
return proto.TaskResult{Error: "загрузка прервана: " + string(gone)}
}
return proto.TaskResult{Error: "загрузка прервана: " + err.Error()}
}
if err := tmp.Truncate(args.Size); err != nil {
return proto.TaskResult{Error: err.Error()}
}
if err := tmp.Close(); err != nil {
return proto.TaskResult{Error: err.Error()}
}
// A file that is being replaced keeps its mode and owner.
mode := os.FileMode(0o644)
if info, err := os.Stat(args.Path); err == nil {
mode = info.Mode().Perm()
if st, ok := info.Sys().(*syscall.Stat_t); ok {
_ = os.Chown(tmp.Name(), int(st.Uid), int(st.Gid))
}
}
if err := os.Chmod(tmp.Name(), mode); err != nil {
return proto.TaskResult{Error: err.Error()}
}
if err := os.Rename(tmp.Name(), args.Path); err != nil {
return proto.TaskResult{Error: err.Error()}
}
done = true
data, _ := json.Marshal(map[string]any{"path": args.Path, "size": args.Size})
return proto.TaskResult{OK: true, Data: data}
}
// doFSSend sends a file to the browser that asked for it.
func (a *Agent) doFSSend(ctx context.Context, env proto.TaskEnvelope) proto.TaskResult {
var args proto.FSXferArgs
if err := json.Unmarshal(env.Args, &args); err != nil {
return proto.TaskResult{Error: err.Error()}
}
if args.Chunk <= 0 {
return proto.TaskResult{Error: "не задан размер куска"}
}
f, err := os.Open(args.Path)
if err != nil {
return proto.TaskResult{Error: err.Error()}
}
defer f.Close()
info, err := f.Stat()
if err != nil {
return proto.TaskResult{Error: err.Error()}
}
if info.IsDir() {
return proto.TaskResult{Error: "это каталог"}
}
size := info.Size()
if size > proto.FSFileCap {
return proto.TaskResult{Error: "файл больше 512 МБ"}
}
sizeHdr := strconv.FormatInt(size, 10)
err = pieces(ctx, pieceCount(size, args.Chunk), func(ctx context.Context, i int) error {
off := int64(i) * int64(args.Chunk)
buf := make([]byte, min(int64(args.Chunk), size-off))
if _, err := f.ReadAt(buf, off); err != nil && err != io.EOF {
return err
}
return a.xferRequest(ctx, func(ctx context.Context) (*http.Request, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, a.xferURL(args.Xfer, i), bytes.NewReader(buf))
if err == nil {
req.Header.Set("X-Size", sizeHdr)
req.Header.Set("Content-Type", "application/octet-stream")
}
return req, err
}, func(*http.Response) error { return nil })
})
if err != nil {
return proto.TaskResult{Error: "передача прервана: " + err.Error()}
}
data, _ := json.Marshal(map[string]any{"path": args.Path, "size": size})
return proto.TaskResult{OK: true, Data: data}
}
// ordered fetches pieces 0..n-1 with up to depth in flight and hands them to
// use one by one in their order: a stream that must be written in sequence
// (stdin over SSH) still gets its pieces fetched ahead.
func ordered(ctx context.Context, n, depth int, fetch func(ctx context.Context, i int) ([]byte, error), use func(i int, b []byte) error) error {
ctx, cancel := context.WithCancel(ctx)
defer cancel()
type got struct {
b []byte
err error
}
slots := make([]chan got, n)
for i := range slots {
slots[i] = make(chan got, 1)
}
sem := make(chan struct{}, depth)
go func() {
for i := 0; i < n; i++ {
select {
case sem <- struct{}{}:
case <-ctx.Done():
return
}
go func(i int) {
b, err := fetch(ctx, i)
slots[i] <- got{b, err}
}(i)
}
}()
for i := 0; i < n; i++ {
var g got
select {
case g = <-slots[i]:
case <-ctx.Done():
return ctx.Err()
}
<-sem
if g.err != nil {
return g.err
}
if err := use(i, g.b); err != nil {
return err
}
}
return nil
}
+176
View File
@@ -0,0 +1,176 @@
package agent
import (
"context"
"log"
"net/http"
"strings"
"sync"
"time"
"github.com/x-grk/longpolling-remote-access/internal/proto"
)
// The server can be reached at two addresses: the main one (behind
// Cloudflare, say) and a direct one that goes to the server itself. The
// direct one comes from the installer or, for agents installed before it was
// set, from the server, and is kept in the state file. When the address in
// use has been out of reach for a while the agent tries the other; on the
// direct one it keeps checking the main one and goes back when it answers.
const (
// switchAfter is how long the address in use may fail before the other
// is tried.
switchAfter = 30 * time.Second
// mainCheck is how often the main address is checked from the direct one.
mainCheck = 5 * time.Minute
)
type hostState struct {
mu sync.Mutex
cur string
downSince time.Time
changed bool // the address changed: the WebSocket goes there at once
}
// host is the server address in use now.
func (a *Agent) host() string {
a.hosts.mu.Lock()
defer a.hosts.mu.Unlock()
if a.hosts.cur == "" {
return a.cfg.Host
}
return a.hosts.cur
}
// fallback is the direct address: the installer's, else the one the server
// told.
func (a *Agent) fallback() string {
if a.cfg.Fallback != "" {
return a.cfg.Fallback
}
a.stateMu.Lock()
defer a.stateMu.Unlock()
return a.state.Fallback
}
// learnAddrs keeps the server's other address from the two it tells: the
// direct one, or the main one when the agent was set to the direct one.
func (a *Agent) learnAddrs(main, direct string) {
main = strings.TrimRight(strings.TrimSpace(main), "/")
direct = strings.TrimRight(strings.TrimSpace(direct), "/")
v := direct
if direct != "" && strings.EqualFold(direct, a.cfg.Host) {
v = main
}
if v == "" || strings.EqualFold(v, a.cfg.Host) || a.cfg.Fallback != "" {
return
}
a.stateMu.Lock()
same := a.state.Fallback == v
if !same {
a.state.Fallback = v
}
a.stateMu.Unlock()
if !same {
log.Printf("fallback address: %s", v)
_ = a.saveState()
}
}
// setHost moves to addr: requests go there from now on, and the WebSocket
// is dropped to reconnect there.
func (a *Agent) setHost(addr string) {
a.hosts.mu.Lock()
if a.hosts.cur == addr || (a.hosts.cur == "" && addr == a.cfg.Host) {
a.hosts.mu.Unlock()
return
}
a.hosts.cur, a.hosts.downSince, a.hosts.changed = addr, time.Time{}, true
a.hosts.mu.Unlock()
log.Printf("server address: %s", addr)
if l := a.link.Load(); l != nil {
l.close()
}
}
// hostChanged reports, once, that the address changed.
func (a *Agent) hostChanged() bool {
a.hosts.mu.Lock()
defer a.hosts.mu.Unlock()
c := a.hosts.changed
a.hosts.changed = false
return c
}
// reached notes the server answered; failed notes it did not, and after
// switchAfter of that moves to the other address.
func (a *Agent) reached() {
a.hosts.mu.Lock()
a.hosts.downSince = time.Time{}
a.hosts.mu.Unlock()
}
func (a *Agent) failed() {
fb := a.fallback()
if fb == "" || fb == a.cfg.Host {
return
}
a.hosts.mu.Lock()
now := time.Now()
if a.hosts.downSince.IsZero() {
a.hosts.downSince = now
a.hosts.mu.Unlock()
return
}
long := now.Sub(a.hosts.downSince) >= switchAfter
cur := a.hosts.cur
a.hosts.mu.Unlock()
if !long {
return
}
next := fb
if cur == fb {
next = a.cfg.Host
}
log.Printf("server out of reach for %s, trying %s", switchAfter, next)
a.setHost(next)
}
// watchMain moves back to the main address once it answers again.
func (a *Agent) watchMain(ctx context.Context) {
t := time.NewTicker(mainCheck)
defer t.Stop()
for {
select {
case <-ctx.Done():
return
case <-t.C:
}
if a.host() == a.cfg.Host {
continue
}
if a.reachableAt(ctx, a.cfg.Host) {
log.Printf("main address answers again")
a.setHost(a.cfg.Host)
}
}
}
// reachableAt reports whether the server answers a quick request at base.
func (a *Agent) reachableAt(ctx context.Context, base string) bool {
ctx, cancel := context.WithTimeout(ctx, 10*time.Second)
defer cancel()
req, _ := http.NewRequestWithContext(ctx, http.MethodGet, base+"/api/agent/config", nil)
a.authHeaders(req)
resp, err := a.http.Do(req)
if err != nil {
return false
}
resp.Body.Close()
if resp.StatusCode < 300 {
a.learnAddrs(resp.Header.Get(proto.MainHeader), resp.Header.Get(proto.FallbackHeader))
return true
}
return false
}
+45
View File
@@ -0,0 +1,45 @@
package agent
import (
"context"
"math/rand"
"sync/atomic"
"testing"
"time"
)
// Pieces fetched side by side are used in their order, never more than
// depth at once in flight.
func TestOrderedKeepsOrder(t *testing.T) {
var inFlight, most int32
var got []int
err := ordered(context.Background(), 20, 3, func(ctx context.Context, i int) ([]byte, error) {
n := atomic.AddInt32(&inFlight, 1)
for {
m := atomic.LoadInt32(&most)
if n <= m || atomic.CompareAndSwapInt32(&most, m, n) {
break
}
}
time.Sleep(time.Duration(rand.Intn(5)) * time.Millisecond)
atomic.AddInt32(&inFlight, -1)
return []byte{byte(i)}, nil
}, func(i int, b []byte) error {
if int(b[0]) != i {
t.Errorf("piece %d carries %d", i, b[0])
}
got = append(got, i)
return nil
})
if err != nil || len(got) != 20 {
t.Fatalf("got %v, %v", got, err)
}
for i, v := range got {
if v != i {
t.Fatalf("order: %v", got)
}
}
if most > 3 || most < 2 {
t.Fatalf("at most %d in flight", most)
}
}
+46 -7
View File
@@ -16,8 +16,8 @@ import (
// runTerminal spawns a PTY and bridges it to the session.
// user is the OS user to run the shell as ("" = the agent's own user).
func (a *Agent) runTerminal(parent context.Context, sid, user string) {
term, err := StartPTY(user)
func (a *Agent) runTerminal(parent context.Context, sid, user, dir string) {
term, err := StartPTY(user, dir)
if err != nil {
log.Printf("term %s: start pty: %v", sid, err)
// Show the reason in the browser terminal before closing it.
@@ -26,12 +26,15 @@ func (a *Agent) runTerminal(parent context.Context, sid, user string) {
return
}
log.Printf("term %s: shell started (user %q)", sid, user)
termsBySID.Store(sid, term)
defer termsBySID.Delete(sid)
a.bridgeStream(parent, sid, term, term.Resize)
log.Printf("term %s: ended", sid)
}
// runForward dials 127.0.0.1:port on the client and bridges it to the session,
// giving the host a reverse TCP tunnel over the same long-poll transport.
// giving the host a reverse TCP tunnel over the agent's link to the server:
// the WebSocket, or the long polls without one.
func (a *Agent) runForward(parent context.Context, sid string, port int) {
d := net.Dialer{Timeout: 10 * time.Second}
conn, err := d.DialContext(parent, "tcp", net.JoinHostPort("127.0.0.1", strconv.Itoa(port)))
@@ -54,6 +57,15 @@ func (a *Agent) bridgeStream(parent context.Context, sid string, rw io.ReadWrite
ctx, cancel := context.WithCancel(parent)
defer cancel()
defer rw.Close()
defer func() {
if l := a.link.Load(); l != nil {
l.unsubscribe(sid)
}
}()
// A session closed from the panel (a tab reloaded, a window shut) ends
// the stream at once, not on its next output: an idle shell would live
// on until the agent stopped.
go func() { <-ctx.Done(); rw.Close() }()
// Downstream: pull input frames and write them to the stream.
go func() {
@@ -81,10 +93,26 @@ func (a *Agent) bridgeStream(parent context.Context, sid string, rw io.ReadWrite
}
}
// pollIn performs one downstream long-poll and applies the frames.
// pollIn takes the next input - pushed on the WebSocket, else by a long
// poll - and applies the frames.
// Returns false when the session is gone and the loop should stop.
func (a *Agent) pollIn(ctx context.Context, sid string, w io.Writer, onResize func(cols, rows uint16)) bool {
req, _ := http.NewRequestWithContext(ctx, http.MethodGet, a.cfg.Host+"/api/agent/term/"+sid+"/in", nil)
// On the WebSocket the input comes pushed; if it drops, by a long poll.
if l := a.link.Load(); l != nil {
if ch := l.subscribe(sid); ch != nil {
select {
case in := <-ch:
if in.gone {
return false
}
return applyFrames(in.frames, w, onResize)
case <-l.done:
case <-ctx.Done():
return false
}
}
}
req, _ := http.NewRequestWithContext(ctx, http.MethodGet, a.host()+"/api/agent/term/"+sid+"/in", nil)
a.authHeaders(req)
resp, err := a.http.Do(req)
if err != nil {
@@ -101,7 +129,13 @@ func (a *Agent) pollIn(ctx context.Context, sid string, w io.Writer, onResize fu
if err := json.NewDecoder(resp.Body).Decode(&batch); err != nil {
return true
}
for _, f := range batch.Frames {
return applyFrames(batch.Frames, w, onResize)
}
// applyFrames writes a session's input to its stream; false once the panel
// closed it.
func applyFrames(frames []proto.InFrame, w io.Writer, onResize func(cols, rows uint16)) bool {
for _, f := range frames {
switch f.Type {
case proto.FrameStdin:
if data, err := b64.DecodeString(f.Data); err == nil {
@@ -120,8 +154,13 @@ func (a *Agent) pollIn(ctx context.Context, sid string, w io.Writer, onResize fu
// postOut sends one upstream message. Returns false if the session is gone.
func (a *Agent) postOut(ctx context.Context, sid string, msg proto.OutMessage) bool {
if l := a.link.Load(); l != nil {
if l.send(proto.WSMsg{T: "out", SID: sid, Out: &msg}) == nil {
return true
}
}
body, _ := json.Marshal(msg)
req, _ := http.NewRequestWithContext(ctx, http.MethodPost, a.cfg.Host+"/api/agent/term/"+sid+"/out", bytes.NewReader(body))
req, _ := http.NewRequestWithContext(ctx, http.MethodPost, a.host()+"/api/agent/term/"+sid+"/out", bytes.NewReader(body))
a.authHeaders(req)
req.Header.Set("Content-Type", "application/json")
resp, err := a.http.Do(req)
+675
View File
@@ -0,0 +1,675 @@
package agent
import (
"archive/tar"
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"io/fs"
"net/http"
"os"
"path"
"path/filepath"
"strconv"
"strings"
"sync"
"sync/atomic"
"syscall"
"time"
"golang.org/x/crypto/ssh"
"github.com/x-grk/longpolling-remote-access/internal/proto"
)
// A file or folder goes to another machine as a tar stream: packed here on
// the fly, then either posted in pieces to a relay on the server for the
// other agent to unpack, or written by SSH straight into tar on that host.
// Nothing is staged on disk on either side.
// shipMeter counts what has gone out, for the panel's progress bar.
type shipMeter struct {
total, done atomic.Int64
files atomic.Int64
note atomic.Value // string
}
func (m *shipMeter) snapshot() proto.ShipProgress {
n, _ := m.note.Load().(string)
return proto.ShipProgress{Total: m.total.Load(), Done: m.done.Load(), Files: int(m.files.Load()), Note: n}
}
// reportShip posts the progress to the job until ctx ends; the server
// answering 410 means the job was cancelled, and cancel is called.
func (a *Agent) reportShip(ctx context.Context, job string, m *shipMeter, cancel func()) {
if job == "" {
return
}
t := time.NewTicker(700 * time.Millisecond)
defer t.Stop()
for {
body, _ := json.Marshal(m.snapshot())
req, _ := http.NewRequestWithContext(ctx, http.MethodPost, a.host()+"/api/agent/fs/job/"+job+"/progress", bytes.NewReader(body))
a.authHeaders(req)
req.Header.Set("Content-Type", "application/json")
if resp, err := a.http.Do(req); err == nil {
io.Copy(io.Discard, resp.Body)
resp.Body.Close()
if resp.StatusCode == http.StatusGone {
cancel()
return
}
}
select {
case <-ctx.Done():
return
case <-t.C:
}
}
}
// shipName checks the name the thing gets at the other end.
func shipName(name string) error {
if name == "" || name == "." || name == ".." || strings.ContainsAny(name, "/\x00") {
return errors.New("недопустимое имя")
}
return nil
}
// treeSize is how many bytes of files are under p.
func treeSize(ctx context.Context, p string) int64 {
var n int64
_ = filepath.WalkDir(p, func(_ string, d fs.DirEntry, err error) error {
if err != nil || ctx.Err() != nil {
return nil
}
if d.Type().IsRegular() {
if info, err := d.Info(); err == nil {
n += info.Size()
}
}
return nil
})
return n
}
// packTar writes src as a tar stream whose entries start at name. Owners are
// left out: the numbers mean other people on another machine.
func packTar(ctx context.Context, w io.Writer, src, name string, m *shipMeter) error {
return packTars(ctx, w, []shipItem{{src, name}}, m)
}
// shipItem is one thing sent: where it is and the name it goes under.
type shipItem struct{ src, name string }
// packTars writes the items as one tar stream, each under its name.
func packTars(ctx context.Context, w io.Writer, items []shipItem, m *shipMeter) error {
tw := tar.NewWriter(w)
for _, it := range items {
if err := packInto(ctx, tw, it.src, it.name, m); err != nil {
return err
}
}
return tw.Close()
}
func packInto(ctx context.Context, tw *tar.Writer, src, name string, m *shipMeter) error {
err := filepath.WalkDir(src, func(p string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if err := ctx.Err(); err != nil {
return err
}
info, err := os.Lstat(p)
if err != nil {
return err
}
link := ""
switch {
case info.Mode()&fs.ModeSymlink != 0:
if link, err = os.Readlink(p); err != nil {
return err
}
case info.IsDir(), info.Mode().IsRegular():
default:
return nil // sockets, devices, pipes: not sent
}
hdr, err := tar.FileInfoHeader(info, link)
if err != nil {
return err
}
rel, _ := filepath.Rel(src, p)
hdr.Name = name
if rel != "." {
hdr.Name = path.Join(name, filepath.ToSlash(rel))
}
if info.IsDir() {
hdr.Name += "/"
}
hdr.Uid, hdr.Gid, hdr.Uname, hdr.Gname = 0, 0, "", ""
hdr.Format = tar.FormatPAX
if err := tw.WriteHeader(hdr); err != nil {
return err
}
if info.Mode().IsRegular() {
f, err := os.Open(p)
if err != nil {
return err
}
_, err = io.CopyN(tw, &countReader{r: f, n: &m.done}, hdr.Size)
f.Close()
if err != nil {
return fmt.Errorf("%s: %v", p, err)
}
}
m.files.Add(1)
return nil
})
return err
}
type countReader struct {
r io.Reader
n *atomic.Int64
}
func (c *countReader) Read(p []byte) (int, error) {
n, err := c.r.Read(p)
c.n.Add(int64(n))
return n, err
}
// doShipOut sends Src to the other machine: by the relay, or by SSH.
func (a *Agent) doShipOut(parent context.Context, env proto.TaskEnvelope) proto.TaskResult {
var args proto.FSShipArgs
if err := json.Unmarshal(env.Args, &args); err != nil {
return proto.TaskResult{Error: err.Error()}
}
names := args.ShipNames()
for _, n := range names {
if err := shipName(n); err != nil {
return proto.TaskResult{Error: err.Error()}
}
}
if len(args.More) > 0 && args.SSH != nil {
return proto.TaskResult{Error: "по SSH-адресу - по одному"}
}
src := filepath.Clean(args.Src)
srcs := []string{args.Src}
for _, p := range args.More {
// The others: of the same folder only.
if path.Dir(path.Clean(p)) != path.Dir(path.Clean(args.Src)) {
return proto.TaskResult{Error: "несколько - только из одной папки"}
}
srcs = append(srcs, p)
}
var conn *sshfsConn
if args.Conn != "" {
// The files are on an SSH host this agent holds a connection to.
c, err := sshfsGet(args.Conn)
if err != nil {
return proto.TaskResult{Error: err.Error()}
}
conn, src = c, path.Clean(args.Src)
} else if _, err := os.Lstat(src); err != nil {
return proto.TaskResult{Error: err.Error()}
}
ctx, cancel := context.WithCancel(parent)
defer cancel()
m := &shipMeter{}
m.note.Store("подсчёт")
go a.reportShip(ctx, args.Job, m, cancel)
var total int64
items := make([]shipItem, len(srcs))
for i, p := range srcs {
if conn != nil {
total += a.remoteSize(conn, path.Clean(p))
} else {
total += treeSize(ctx, filepath.Clean(p))
}
items[i] = shipItem{filepath.Clean(p), names[i]}
}
m.total.Store(total)
pack := func(w io.Writer) error { return packTars(ctx, w, items, m) }
if conn != nil {
pack = func(w io.Writer) error { return a.packRemote(ctx, w, conn, srcs, names, m) }
}
var err error
if args.SSH != nil {
err = a.shipSSH(ctx, args, src, m)
} else {
m.note.Store("")
err = a.shipRelay(ctx, args, pack)
}
if err != nil {
if ctx.Err() != nil && parent.Err() == nil {
err = errors.New("отменено")
}
return proto.TaskResult{Error: err.Error()}
}
res := map[string]any{"files": m.files.Load(), "bytes": m.done.Load()}
if args.Move && args.SSH != nil {
if err := os.RemoveAll(src); err != nil {
return proto.TaskResult{Error: "отправлено, но здесь не удалилось: " + err.Error()}
}
}
data, _ := json.Marshal(res)
return proto.TaskResult{OK: true, Data: data}
}
func (a *Agent) relayURL(id string, i int) string {
return a.host() + "/api/agent/fs/relay/" + id + "/" + strconv.Itoa(i)
}
// shipRelay posts the tar stream in pieces, a few at once; the last carries
// X-Last.
func (a *Agent) shipRelay(ctx context.Context, args proto.FSShipArgs, pack func(io.Writer) error) error {
if args.Relay == "" || args.Chunk <= 0 {
return errors.New("не задана передача")
}
ctx, cancel := context.WithCancel(ctx)
defer cancel()
pr, pw := io.Pipe()
go func() { pw.CloseWithError(pack(pw)) }()
defer pr.Close()
type piece struct {
i int
data []byte
last bool
}
ch := make(chan piece, 1)
var (
mu sync.Mutex
first error
wg sync.WaitGroup
)
setErr := func(err error) {
mu.Lock()
if first == nil {
first = err
cancel()
}
mu.Unlock()
}
for w := 0; w < xferWorkers; w++ {
wg.Add(1)
go func() {
defer wg.Done()
for p := range ch {
err := a.xferRequest(ctx, func(ctx context.Context) (*http.Request, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, a.relayURL(args.Relay, p.i), bytes.NewReader(p.data))
if err == nil {
req.Header.Set("Content-Type", "application/octet-stream")
if p.last {
req.Header.Set("X-Last", "1")
}
}
return req, err
}, func(*http.Response) error { return nil })
if err != nil {
setErr(err)
}
}
}()
}
for i := 0; ; i++ {
buf := make([]byte, args.Chunk)
n, err := io.ReadFull(pr, buf)
last := err == io.EOF || err == io.ErrUnexpectedEOF
if err != nil && !last {
setErr(err)
break
}
select {
case ch <- piece{i, buf[:n], last}:
case <-ctx.Done():
}
if last || ctx.Err() != nil {
break
}
}
close(ch)
wg.Wait()
if first != nil {
var gone errXferGone
if errors.As(first, &gone) {
return errors.New(string(gone))
}
return first
}
return ctx.Err()
}
// doShipIn takes the tar stream from the relay and unpacks it into Dir.
func (a *Agent) doShipIn(ctx context.Context, env proto.TaskEnvelope) proto.TaskResult {
var args proto.FSShipArgs
if err := json.Unmarshal(env.Args, &args); err != nil {
return proto.TaskResult{Error: err.Error()}
}
names := args.ShipNames()
for _, n := range names {
if err := shipName(n); err != nil {
return proto.TaskResult{Error: err.Error()}
}
}
dir := filepath.Clean(args.Dir)
var conn *sshfsConn
if args.Conn != "" {
// Into an SSH host this agent holds a connection to.
c, err := sshfsGet(args.Conn)
if err != nil {
return proto.TaskResult{Error: err.Error()}
}
conn, dir = c, path.Clean(args.Dir)
} else {
for _, n := range names {
if _, err := os.Lstat(filepath.Join(dir, n)); err == nil && args.Conflict != "replace" && args.Conflict != "skip" {
return proto.TaskResult{Error: filepath.Join(dir, n) + " уже существует"}
}
}
if err := os.MkdirAll(dir, 0o755); err != nil {
return proto.TaskResult{Error: err.Error()}
}
}
ctx, cancel := context.WithCancel(ctx)
defer cancel()
pr, pw := io.Pipe()
go func() {
for i := 0; ; i++ {
var data []byte
var last bool
err := a.xferRequest(ctx, func(ctx context.Context) (*http.Request, error) {
return http.NewRequestWithContext(ctx, http.MethodGet, a.relayURL(args.Relay, i), nil)
}, func(resp *http.Response) error {
b, err := io.ReadAll(io.LimitReader(resp.Body, int64(args.Chunk)+1))
data, last = b, resp.Header.Get("X-Last") != ""
return err
})
if err == nil {
_, err = pw.Write(data)
}
if err != nil {
pw.CloseWithError(err)
return
}
if last {
pw.Close()
return
}
}
}()
if conn != nil {
skipped, err := a.unpackRemote(conn, pr, dir, names, args.Conflict)
if err == nil {
_, _ = io.Copy(io.Discard, pr)
}
pr.CloseWithError(errors.New("распаковка закончена"))
if err != nil {
return proto.TaskResult{Error: err.Error()}
}
res := map[string]int{}
if skipped > 0 {
res["skipped"] = skipped
}
data, _ := json.Marshal(res)
return proto.TaskResult{OK: true, Data: data}
}
allowed := map[string]bool{}
for _, n := range names {
allowed[n] = true
}
u := &unpacker{dir: dir, names: allowed, conflict: args.Conflict}
if os.Geteuid() == 0 {
// New things belong to whoever owns the folder they land in.
if info, err := os.Stat(dir); err == nil {
if st, ok := info.Sys().(*syscall.Stat_t); ok {
u.uid, u.gid, u.chown = int(st.Uid), int(st.Gid), true
}
}
}
err := u.unpack(tar.NewReader(pr))
if err == nil {
// Whatever follows the archive's end is read off, so the sender
// finishes.
_, err = io.Copy(io.Discard, pr)
}
pr.CloseWithError(errors.New("распаковка закончена"))
if err != nil {
var gone errXferGone
if errors.As(err, &gone) {
err = errors.New(string(gone))
}
return proto.TaskResult{Error: err.Error()}
}
data, _ := json.Marshal(map[string]int{"done": u.done, "skipped": u.skipped})
return proto.TaskResult{OK: true, Data: data}
}
// unpacker writes a tar stream into dir, every entry under name, following
// the conflict rule for what is there already: folders merge, and files are
// replaced or kept.
type unpacker struct {
dir, conflict string
names map[string]bool // the top entries the stream may carry
uid, gid int
chown bool
done, skipped int
}
func (u *unpacker) unpack(tr *tar.Reader) error {
var skipUnder string
for {
hdr, err := tr.Next()
if err == io.EOF {
return nil
}
if err != nil {
return err
}
clean := path.Clean(hdr.Name)
top, _, _ := strings.Cut(clean, "/")
if !u.names[top] || clean == ".." || strings.HasPrefix(clean, "../") {
return fmt.Errorf("в архиве чужой путь %q", hdr.Name)
}
target := filepath.Join(u.dir, filepath.FromSlash(clean))
if skipUnder != "" && strings.HasPrefix(target, skipUnder) {
continue
}
have, err := os.Lstat(target)
exists := err == nil
if exists && !(hdr.Typeflag == tar.TypeDir && have.IsDir()) {
switch u.conflict {
case "skip":
u.skipped++
if hdr.Typeflag == tar.TypeDir {
skipUnder = target + string(filepath.Separator)
}
continue
case "replace":
if have.IsDir() || hdr.Typeflag != tar.TypeReg {
if err := os.RemoveAll(target); err != nil {
return err
}
}
default:
return fmt.Errorf("%s уже существует", target)
}
}
mode := os.FileMode(hdr.Mode).Perm()
switch hdr.Typeflag {
case tar.TypeDir:
if !exists || !have.IsDir() {
if err := os.MkdirAll(target, mode|0o700); err != nil {
return err
}
_ = os.Chmod(target, mode)
u.own(target)
}
case tar.TypeReg:
if err := u.writeFile(target, tr, mode, hdr.ModTime); err != nil {
return err
}
case tar.TypeSymlink:
if err := os.Symlink(hdr.Linkname, target); err != nil {
return err
}
u.own(target)
default:
continue
}
u.done++
}
}
// writeFile writes next to the target first and puts it in place when all of
// it is there.
func (u *unpacker) writeFile(target string, r io.Reader, mode os.FileMode, mtime time.Time) error {
tmp, err := os.CreateTemp(filepath.Dir(target), "."+filepath.Base(target)+".lpra-*")
if err != nil {
return err
}
if _, err := io.Copy(tmp, r); err != nil {
tmp.Close()
os.Remove(tmp.Name())
return err
}
if err := tmp.Close(); err != nil {
os.Remove(tmp.Name())
return err
}
_ = os.Chmod(tmp.Name(), mode)
_ = os.Chtimes(tmp.Name(), mtime, mtime)
u.own(tmp.Name())
if err := os.Rename(tmp.Name(), target); err != nil {
os.Remove(tmp.Name())
return err
}
return nil
}
func (u *unpacker) own(p string) {
if u.chown {
_ = os.Lchown(p, u.uid, u.gid)
}
}
// shq quotes s for a POSIX shell.
func shq(s string) string { return "'" + strings.ReplaceAll(s, "'", `'\''`) + "'" }
// posix wraps a command for the host's sh: the user's login shell gets one
// word to hand on, so fish or csh there does not matter.
func posix(cmd string) string {
// csh reads "!" as history even inside single quotes; '\!' is a plain
// "!" for it and for sh alike.
return "sh -c " + strings.ReplaceAll(shq(cmd), "!", `'\!'`)
}
// shipSSH writes the tar stream into tar on the SSH host, in Dir.
func (a *Agent) shipSSH(ctx context.Context, args proto.FSShipArgs, src string, m *shipMeter) error {
client, _, err := a.sshConnect(ctx, *args.SSH, func(_, note string) { m.note.Store(note) })
if err != nil {
return err
}
defer client.Close()
go func() { <-ctx.Done(); client.Close() }()
dir := args.Dir
if dir == "" {
dir = "."
}
target := strings.TrimSuffix(dir, "/") + "/" + args.Name
// What tar is there, and whether the target is.
m.note.Store("проверка на той стороне")
out, err := sshRun(client, "command -v tar >/dev/null 2>&1 || echo LPRA_NOTAR; tar --version 2>/dev/null | head -n 1; test -e "+shq(target)+" -o -L "+shq(target)+" && echo LPRA_EXISTS")
if err != nil {
return err
}
if strings.Contains(out, "LPRA_NOTAR") {
return errors.New("на той стороне нет tar")
}
if strings.Contains(out, "LPRA_EXISTS") && args.Conflict != "replace" && args.Conflict != "skip" {
return errors.New(target + " там уже существует")
}
gnu := strings.Contains(out, "GNU tar")
flags := ""
switch args.Conflict {
case "skip":
if gnu {
flags = " --skip-old-files"
} else {
flags = " -k"
}
case "replace":
if gnu {
flags = " --overwrite"
}
}
sess, err := client.NewSession()
if err != nil {
return err
}
defer sess.Close()
var stderr bytes.Buffer
sess.Stderr = &limitBuf{b: &stderr, max: 4096}
stdin, err := sess.StdinPipe()
if err != nil {
return err
}
cmd := "mkdir -p -- " + shq(dir) + " && cd -- " + shq(dir) + " && tar -xf -" + flags
if err := sess.Start(posix(cmd)); err != nil {
return err
}
m.note.Store("")
packErr := packTar(ctx, stdin, src, args.Name, m)
stdin.Close()
waitErr := sess.Wait()
msg := strings.TrimSpace(stderr.String())
if packErr != nil {
if msg != "" {
return fmt.Errorf("%v (%s)", packErr, msg)
}
return packErr
}
if waitErr != nil {
var ee *ssh.ExitError
if errors.As(waitErr, &ee) && msg != "" {
return errors.New("tar на той стороне: " + msg)
}
return waitErr
}
return nil
}
// sshRun runs a command and returns its output.
func sshRun(c *ssh.Client, cmd string) (string, error) {
s, err := c.NewSession()
if err != nil {
return "", err
}
defer s.Close()
out, err := s.Output(posix(cmd))
var ee *ssh.ExitError
if errors.As(err, &ee) {
err = nil // the checks may exit non-zero; what they print is the answer
}
return string(out), err
}
type limitBuf struct {
b *bytes.Buffer
max int
}
func (l *limitBuf) Write(p []byte) (int, error) {
if room := l.max - l.b.Len(); room > 0 {
l.b.Write(p[:min(len(p), room)])
}
return len(p), nil
}
+66
View File
@@ -0,0 +1,66 @@
package agent
import (
"archive/tar"
"bytes"
"context"
"os"
"path/filepath"
"strings"
"testing"
)
// Several items of one folder go as one tar and land side by side; an entry
// of none of them is refused.
func TestShipSeveralInOneTar(t *testing.T) {
src, dst := t.TempDir(), t.TempDir()
_ = os.WriteFile(filepath.Join(src, "a.txt"), []byte("A"), 0o644)
_ = os.MkdirAll(filepath.Join(src, "b"), 0o755)
_ = os.WriteFile(filepath.Join(src, "b", "c.txt"), []byte("C"), 0o644)
var buf bytes.Buffer
items := []shipItem{{filepath.Join(src, "a.txt"), "a.txt"}, {filepath.Join(src, "b"), "b"}}
if err := packTars(context.Background(), &buf, items, &shipMeter{}); err != nil {
t.Fatal(err)
}
u := &unpacker{dir: dst, names: map[string]bool{"a.txt": true, "b": true}}
if err := u.unpack(tar.NewReader(&buf)); err != nil {
t.Fatal(err)
}
if b, _ := os.ReadFile(filepath.Join(dst, "b", "c.txt")); string(b) != "C" {
t.Fatalf("b/c.txt: %q", b)
}
if b, _ := os.ReadFile(filepath.Join(dst, "a.txt")); string(b) != "A" {
t.Fatalf("a.txt: %q", b)
}
buf.Reset()
_ = packTars(context.Background(), &buf, items, &shipMeter{})
u = &unpacker{dir: t.TempDir(), names: map[string]bool{"a.txt": true}}
if err := u.unpack(tar.NewReader(&buf)); err == nil || !strings.Contains(err.Error(), "чужой путь") {
t.Fatalf("b let in: %v", err)
}
}
func TestRenameTarSeveral(t *testing.T) {
var src bytes.Buffer
tw := tar.NewWriter(&src)
for _, n := range []string{"a", "b/x", "c", "._a"} {
_ = tw.WriteHeader(&tar.Header{Name: n, Mode: 0o644, Typeflag: tar.TypeReg})
}
_ = tw.Close()
var out bytes.Buffer
if err := renameTar(&out, &src, map[string]string{"a": "a", "b": "b2"}); err != nil {
t.Fatal(err)
}
tr := tar.NewReader(&out)
var got []string
for {
h, err := tr.Next()
if err != nil {
break
}
got = append(got, h.Name)
}
if strings.Join(got, ",") != "a,b2/x" {
t.Fatalf("entries: %v", got)
}
}
+167
View File
@@ -0,0 +1,167 @@
package agent
import (
"context"
"crypto/rand"
"encoding/json"
"errors"
"io"
"net/http"
"time"
"github.com/x-grk/longpolling-remote-access/internal/proto"
)
const (
// speedWindow is how long each direction is measured for, unless the
// server asks for another window; speedWindowMax bounds what it may ask.
speedWindow = 3 * time.Second
speedWindowMax = 30 * time.Second
// speedWait is how long the download may take to start: in a relayed test
// its bytes come from the browser.
speedWait = 10 * time.Second
// speedUpCap bounds the upload, whatever the link does in the window.
speedUpCap = 48 << 20
)
// doSpeed measures the agent's HTTP link to the server: round-trip time of an
// empty request, then how much flows each way in the window. In a relayed
// test the server pipes both streams to the browser, so they run end to end.
func (a *Agent) doSpeed(ctx context.Context, env proto.TaskEnvelope) proto.TaskResult {
var args proto.SysSpeedArgs
if len(env.Args) > 0 {
_ = json.Unmarshal(env.Args, &args)
}
window := speedWindow
if args.WindowMs > 0 {
window = min(time.Duration(args.WindowMs)*time.Millisecond, speedWindowMax)
}
res, err := a.measureSpeed(ctx, args.Relay, window)
if err != nil {
return proto.TaskResult{Error: err.Error()}
}
data, _ := json.Marshal(res)
return proto.TaskResult{OK: true, Data: data}
}
func (a *Agent) measureSpeed(ctx context.Context, relay bool, window time.Duration) (proto.SysSpeedResult, error) {
res := proto.SysSpeedResult{Relay: relay}
streams := "/api/agent/speed"
if relay {
streams += "/relay"
}
// Ping: the first request may pay for a new connection, so keep the best.
for i := 0; i < 4; i++ {
start := time.Now()
resp, err := a.speedRequest(ctx, http.MethodGet, "/api/agent/speed/down?bytes=0", nil)
if err != nil {
return res, err
}
io.Copy(io.Discard, resp.Body)
resp.Body.Close()
ms := float64(time.Since(start).Microseconds()) / 1000
if i == 0 || ms < res.PingMs {
res.PingMs = ms
}
}
down, err := a.speedDown(ctx, streams+"/down", window)
if err != nil {
return res, err
}
res.DownBps = down
// Up: send until the window closes or the cap is hit.
body := &speedSource{deadline: time.Now().Add(window), left: speedUpCap}
start := time.Now()
resp, err := a.speedRequest(ctx, http.MethodPost, streams+"/up", body)
if err != nil {
return res, err
}
io.Copy(io.Discard, resp.Body)
resp.Body.Close()
res.UpBps = bps(int64(speedUpCap-body.left), time.Since(start))
return res, nil
}
// speedDown reads what the server streams for the window, counted from the
// first byte, and returns the rate.
func (a *Agent) speedDown(ctx context.Context, path string, window time.Duration) (float64, error) {
ctx, cancel := context.WithCancel(ctx)
defer cancel()
t := time.AfterFunc(speedWait, cancel)
defer t.Stop()
resp, err := a.speedRequest(ctx, http.MethodGet, path, nil)
if err != nil {
return 0, err
}
defer resp.Body.Close()
var (
n int64
start time.Time
buf = make([]byte, 64<<10)
)
for {
k, err := resp.Body.Read(buf)
if k > 0 && start.IsZero() {
start = time.Now()
t.Reset(window)
}
n += int64(k)
if err != nil {
break
}
}
if start.IsZero() {
return 0, errors.New("нет данных для замера")
}
return bps(n, time.Since(start)), nil
}
func (a *Agent) speedRequest(ctx context.Context, method, path string, body io.Reader) (*http.Response, error) {
req, err := http.NewRequestWithContext(ctx, method, a.host()+path, body)
if err != nil {
return nil, err
}
a.authHeaders(req)
resp, err := a.http.Do(req)
if err != nil {
return nil, err
}
if resp.StatusCode != http.StatusOK && resp.StatusCode != http.StatusNoContent {
resp.Body.Close()
return nil, errStatus(resp.StatusCode)
}
return resp, nil
}
func bps(n int64, d time.Duration) float64 {
if d <= 0 {
return 0
}
return float64(n) * 8 / d.Seconds()
}
// speedSource is an upload body of incompressible bytes that ends at the
// deadline or after `left` bytes.
type speedSource struct {
deadline time.Time
left int
block []byte
}
func (s *speedSource) Read(p []byte) (int, error) {
if s.left <= 0 || !time.Now().Before(s.deadline) {
return 0, io.EOF
}
if s.block == nil {
s.block = make([]byte, 64<<10)
rand.Read(s.block)
}
n := copy(p, s.block)
if n > s.left {
n = s.left
}
s.left -= n
return n, nil
}
+512
View File
@@ -0,0 +1,512 @@
package agent
import (
"context"
"database/sql"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"net/url"
"os"
"strconv"
"strings"
"time"
_ "modernc.org/sqlite"
"github.com/x-grk/longpolling-remote-access/internal/proto"
)
// SQLite databases are read and changed here, through SQLite itself, so a
// database that is in use keeps its locks and journal: a value is set with an
// UPDATE, not by writing the file over.
// dbCell is one value as the panel shows and sends it: t is "i" (integer),
// "r" (real), "s" (text), "b" (blob, v its size, not editable) or "n" (NULL).
type dbCell struct {
T string `json:"t"`
V string `json:"v,omitempty"`
}
func cellOf(v any) dbCell {
switch x := v.(type) {
case nil:
return dbCell{T: "n"}
case int64:
return dbCell{T: "i", V: strconv.FormatInt(x, 10)}
case float64:
return dbCell{T: "r", V: strconv.FormatFloat(x, 'g', -1, 64)}
case string:
return dbCell{T: "s", V: x}
case []byte:
return dbCell{T: "b", V: strconv.Itoa(len(x))}
case bool:
if x {
return dbCell{T: "i", V: "1"}
}
return dbCell{T: "i", V: "0"}
case time.Time:
return dbCell{T: "s", V: x.Format(time.RFC3339Nano)}
}
return dbCell{T: "s", V: fmt.Sprint(v)}
}
// value is what a cell becomes for SQLite.
func (c dbCell) value() (any, error) {
switch c.T {
case "n":
return nil, nil
case "i":
n, err := strconv.ParseInt(strings.TrimSpace(c.V), 10, 64)
if err != nil {
return nil, fmt.Errorf("%q - не целое число", c.V)
}
return n, nil
case "r":
f, err := strconv.ParseFloat(strings.TrimSpace(c.V), 64)
if err != nil {
return nil, fmt.Errorf("%q - не число", c.V)
}
return f, nil
case "x":
b, err := hex.DecodeString(strings.ReplaceAll(c.V, " ", ""))
if err != nil {
return nil, errors.New("blob - нужны шестнадцатеричные байты")
}
return b, nil
}
return c.V, nil
}
type dbArgs struct {
Path string `json:"path"`
Table string `json:"table,omitempty"`
Offset int `json:"offset,omitempty"`
Limit int `json:"limit,omitempty"`
Key json.RawMessage `json:"key,omitempty"`
Keys []json.RawMessage `json:"keys,omitempty"`
Column string `json:"column,omitempty"`
Value *dbCell `json:"value,omitempty"`
Values map[string]dbCell `json:"values,omitempty"`
SQL string `json:"sql,omitempty"`
}
func dbOpen(path string) (*sql.DB, error) {
info, err := os.Stat(path)
if err != nil {
return nil, err
}
if info.IsDir() {
return nil, errors.New("это каталог")
}
u := url.URL{Scheme: "file", Path: path}
db, err := sql.Open("sqlite", u.String()+"?mode=rw&_pragma=busy_timeout(5000)")
if err != nil {
return nil, err
}
db.SetMaxOpenConns(1)
return db, nil
}
func qid(name string) string { return `"` + strings.ReplaceAll(name, `"`, `""`) + `"` }
// dbTable is a table's shape: its columns, and how a row is found again -
// by rowid, by its primary key (a WITHOUT ROWID table), or not at all (a
// view: shown, not changed).
type dbTable struct {
Name string `json:"name"`
Type string `json:"type"`
Columns []dbColumn `json:"columns"`
Key string `json:"key"` // "rowid", "pk" or ""
// Alias is the INTEGER PRIMARY KEY column: another name for the rowid.
Alias string `json:"alias,omitempty"`
pk []string
}
type dbColumn struct {
Name string `json:"name"`
Type string `json:"type"`
NotNull bool `json:"notnull,omitempty"`
PK bool `json:"pk,omitempty"`
Default string `json:"default,omitempty"`
}
func dbTableInfo(ctx context.Context, db *sql.DB, name string) (*dbTable, error) {
t := &dbTable{Name: name}
if err := db.QueryRowContext(ctx, `SELECT type FROM sqlite_master WHERE name = ? AND type IN ('table','view')`, name).Scan(&t.Type); err != nil {
if errors.Is(err, sql.ErrNoRows) {
return nil, fmt.Errorf("нет таблицы %s", name)
}
return nil, err
}
rows, err := db.QueryContext(ctx, `SELECT name, type, "notnull", dflt_value, pk FROM pragma_table_info(?)`, name)
if err != nil {
return nil, err
}
defer rows.Close()
type pkc struct {
n int
col string
}
var pks []pkc
for rows.Next() {
var c dbColumn
var notnull, pk int
var dflt sql.NullString
if err := rows.Scan(&c.Name, &c.Type, &notnull, &dflt, &pk); err != nil {
return nil, err
}
c.NotNull, c.PK, c.Default = notnull == 1, pk > 0, dflt.String
if pk > 0 {
pks = append(pks, pkc{pk, c.Name})
}
t.Columns = append(t.Columns, c)
}
if err := rows.Err(); err != nil {
return nil, err
}
for i := 1; i <= len(pks); i++ {
for _, p := range pks {
if p.n == i {
t.pk = append(t.pk, p.col)
}
}
}
if t.Type == "table" {
if r, err := db.QueryContext(ctx, `SELECT rowid FROM `+qid(name)+` LIMIT 0`); err == nil {
r.Close()
t.Key = "rowid"
if len(t.pk) == 1 {
for _, c := range t.Columns {
if c.Name == t.pk[0] && strings.EqualFold(strings.TrimSpace(c.Type), "INTEGER") {
t.Alias = c.Name
}
}
}
} else if len(t.pk) > 0 {
t.Key = "pk"
}
}
return t, nil
}
// where is the condition that finds the row by its key: a rowid, or the
// primary key's cells in order.
func (t *dbTable) where(key json.RawMessage) (string, []any, error) {
switch t.Key {
case "rowid":
var id int64
if err := json.Unmarshal(key, &id); err != nil {
var s string
if json.Unmarshal(key, &s) != nil {
return "", nil, errors.New("плохой ключ строки")
}
if id, err = strconv.ParseInt(s, 10, 64); err != nil {
return "", nil, errors.New("плохой ключ строки")
}
}
return "rowid = ?", []any{id}, nil
case "pk":
var cells []dbCell
if err := json.Unmarshal(key, &cells); err != nil || len(cells) != len(t.pk) {
return "", nil, errors.New("плохой ключ строки")
}
parts, args := make([]string, len(t.pk)), make([]any, len(t.pk))
for i, col := range t.pk {
v, err := cells[i].value()
if err != nil {
return "", nil, err
}
parts[i], args[i] = qid(col)+" IS ?", v
}
return strings.Join(parts, " AND "), args, nil
}
return "", nil, errors.New("строки этой таблицы нельзя менять: нет ни rowid, ни первичного ключа")
}
func (t *dbTable) column(name string) bool {
for _, c := range t.Columns {
if c.Name == name {
return true
}
}
return false
}
// dbTask runs one database task.
func dbTask(env proto.TaskEnvelope) (any, error) {
var a dbArgs
if err := json.Unmarshal(env.Args, &a); err != nil {
return nil, err
}
db, err := dbOpen(a.Path)
if err != nil {
return nil, err
}
defer db.Close()
ctx, cancel := context.WithTimeout(context.Background(), 50*time.Second)
defer cancel()
if env.Type == proto.TaskDBTables {
rows, err := db.QueryContext(ctx, `SELECT name, type FROM sqlite_master WHERE type IN ('table','view') AND name NOT LIKE 'sqlite_%' ORDER BY type, name`)
if err != nil {
return nil, dbErr(err)
}
defer rows.Close()
type item struct {
Name string `json:"name"`
Type string `json:"type"`
}
out := []item{}
for rows.Next() {
var it item
if err := rows.Scan(&it.Name, &it.Type); err != nil {
return nil, err
}
out = append(out, it)
}
return map[string]any{"tables": out}, rows.Err()
}
if env.Type == proto.TaskDBSQL {
return dbSQL(ctx, db, a.SQL)
}
t, err := dbTableInfo(ctx, db, a.Table)
if err != nil {
return nil, dbErr(err)
}
switch env.Type {
case proto.TaskDBRows:
return dbRows(ctx, db, t, a.Offset, a.Limit)
case proto.TaskDBSet:
if !t.column(a.Column) || a.Value == nil {
return nil, errors.New("нет такого столбца")
}
cond, args, err := t.where(a.Key)
if err != nil {
return nil, err
}
v, err := a.Value.value()
if err != nil {
return nil, err
}
res, err := db.ExecContext(ctx, `UPDATE `+qid(t.Name)+` SET `+qid(a.Column)+` = ? WHERE `+cond, append([]any{v}, args...)...)
if err != nil {
return nil, dbErr(err)
}
if n, _ := res.RowsAffected(); n != 1 {
return nil, fmt.Errorf("изменено строк: %d - строку могли удалить или поменять", n)
}
// A key column changed: the row is found by its new key now.
out := map[string]any{}
switch {
case t.Key == "rowid" && a.Column == t.Alias:
args = []any{v}
out["key"] = a.Value.V
case t.Key == "pk":
var cells []dbCell
_ = json.Unmarshal(a.Key, &cells)
for i, col := range t.pk {
if col == a.Column {
args[i], cells[i] = v, *a.Value
out["key"] = cells
}
}
}
// What SQLite stored, after its type affinity.
var got any
_ = db.QueryRowContext(ctx, `SELECT `+qid(a.Column)+` FROM `+qid(t.Name)+` WHERE `+cond, args...).Scan(&got)
out["value"] = cellOf(got)
if k, ok := out["key"].([]dbCell); ok {
for i, col := range t.pk {
if col == a.Column {
k[i] = cellOf(got)
}
}
}
return out, nil
case proto.TaskDBAdd:
if t.Type != "table" {
return nil, errors.New("в представление строки не добавляются")
}
var cols, marks []string
var vals []any
for name, c := range a.Values {
if !t.column(name) {
return nil, fmt.Errorf("нет столбца %s", name)
}
v, err := c.value()
if err != nil {
return nil, err
}
cols, marks, vals = append(cols, qid(name)), append(marks, "?"), append(vals, v)
}
q := `INSERT INTO ` + qid(t.Name) + ` DEFAULT VALUES`
if len(cols) > 0 {
q = `INSERT INTO ` + qid(t.Name) + ` (` + strings.Join(cols, ", ") + `) VALUES (` + strings.Join(marks, ", ") + `)`
}
res, err := db.ExecContext(ctx, q, vals...)
if err != nil {
return nil, dbErr(err)
}
id, _ := res.LastInsertId()
return map[string]any{"rowid": id}, nil
case proto.TaskDBDel:
tx, err := db.BeginTx(ctx, nil)
if err != nil {
return nil, dbErr(err)
}
defer tx.Rollback()
n := int64(0)
for _, k := range a.Keys {
cond, args, err := t.where(k)
if err != nil {
return nil, err
}
res, err := tx.ExecContext(ctx, `DELETE FROM `+qid(t.Name)+` WHERE `+cond, args...)
if err != nil {
return nil, dbErr(err)
}
c, _ := res.RowsAffected()
n += c
}
if err := tx.Commit(); err != nil {
return nil, dbErr(err)
}
return map[string]any{"deleted": n}, nil
}
return nil, errors.New("неизвестная операция")
}
// dbRows is a page of a table's rows, with each row's key.
func dbRows(ctx context.Context, db *sql.DB, t *dbTable, offset, limit int) (any, error) {
if limit <= 0 || limit > 1000 {
limit = 200
}
var total int64
_ = db.QueryRowContext(ctx, `SELECT COUNT(*) FROM `+qid(t.Name)).Scan(&total)
sel := make([]string, 0, len(t.Columns)+1)
if t.Key == "rowid" {
sel = append(sel, "rowid")
}
for _, c := range t.Columns {
sel = append(sel, qid(c.Name))
}
order := ""
switch t.Key {
case "rowid":
order = " ORDER BY rowid"
case "pk":
parts := make([]string, len(t.pk))
for i, c := range t.pk {
parts[i] = qid(c)
}
order = " ORDER BY " + strings.Join(parts, ", ")
}
rows, err := db.QueryContext(ctx, `SELECT `+strings.Join(sel, ", ")+` FROM `+qid(t.Name)+order+` LIMIT ? OFFSET ?`, limit, offset)
if err != nil {
return nil, dbErr(err)
}
defer rows.Close()
type row struct {
K any `json:"k,omitempty"`
V []dbCell `json:"v"`
}
out := []row{}
n := len(sel)
for rows.Next() {
vals := make([]any, n)
ptrs := make([]any, n)
for i := range vals {
ptrs[i] = &vals[i]
}
if err := rows.Scan(ptrs...); err != nil {
return nil, err
}
r := row{}
if t.Key == "rowid" {
r.K = cellOf(vals[0]).V
vals = vals[1:]
}
for _, v := range vals {
r.V = append(r.V, cellOf(v))
}
if t.Key == "pk" {
key := make([]dbCell, len(t.pk))
for i, pk := range t.pk {
for j, c := range t.Columns {
if c.Name == pk {
key[i] = r.V[j]
}
}
}
r.K = key
}
out = append(out, r)
}
return map[string]any{"table": t, "total": total, "offset": offset, "rows": out}, rows.Err()
}
// dbSQL runs what was typed: a query gives up to 500 rows, anything else
// how many rows it changed.
func dbSQL(ctx context.Context, db *sql.DB, q string) (any, error) {
q = strings.TrimSpace(q)
if q == "" {
return nil, errors.New("пустой запрос")
}
head := strings.ToLower(strings.Fields(q)[0])
if head == "select" || head == "pragma" || head == "with" || head == "explain" || head == "values" {
rows, err := db.QueryContext(ctx, q)
if err != nil {
return nil, dbErr(err)
}
defer rows.Close()
cols, _ := rows.Columns()
out := [][]dbCell{}
more := false
for rows.Next() {
if len(out) == 500 {
more = true
break
}
vals := make([]any, len(cols))
ptrs := make([]any, len(cols))
for i := range vals {
ptrs[i] = &vals[i]
}
if err := rows.Scan(ptrs...); err != nil {
return nil, err
}
r := make([]dbCell, len(cols))
for i, v := range vals {
r[i] = cellOf(v)
}
out = append(out, r)
}
return map[string]any{"columns": cols, "rows": out, "more": more}, dbErr(rows.Err())
}
res, err := db.ExecContext(ctx, q)
if err != nil {
return nil, dbErr(err)
}
n, _ := res.RowsAffected()
return map[string]any{"changed": n}, nil
}
// dbErr says the common SQLite troubles in words.
func dbErr(err error) error {
if err == nil {
return nil
}
m := err.Error()
switch {
case strings.Contains(m, "file is not a database"):
return errors.New("это не база SQLite")
case strings.Contains(m, "database is locked"), strings.Contains(m, "SQLITE_BUSY"):
return errors.New("база занята другой программой - попробуйте ещё раз")
case strings.Contains(m, "readonly"):
return errors.New("база только для чтения (нет прав на файл или его папку)")
}
return err
}
+250
View File
@@ -0,0 +1,250 @@
package agent
import (
"bufio"
"context"
"errors"
"fmt"
"io"
"log"
"net"
"os"
"path/filepath"
"strconv"
"strings"
"sync"
"time"
"golang.org/x/crypto/ssh"
"github.com/x-grk/longpolling-remote-access/internal/proto"
)
// An SSH terminal: the agent connects to another host by SSH and bridges the
// remote shell to the session, as it does a local one. The panel is told how
// the connection goes (connecting, signing in, in) to show it.
// sshKnownHosts holds the host keys seen, next to the agent's state: a host
// is trusted the first time, and a changed key is refused.
func (a *Agent) sshKnownHosts() string {
return filepath.Join(filepath.Dir(a.cfg.StatePath), "ssh_known_hosts")
}
var knownHostsMu sync.Mutex
// hostKeyCheck trusts a new host and remembers its key; a host whose key
// has changed since is refused, as ssh would.
func (a *Agent) hostKeyCheck(seen *string) ssh.HostKeyCallback {
return func(hostname string, _ net.Addr, key ssh.PublicKey) error {
knownHostsMu.Lock()
defer knownHostsMu.Unlock()
fp := ssh.FingerprintSHA256(key)
*seen = fp
path := a.sshKnownHosts()
line := hostname + " " + key.Type() + " "
if f, err := os.Open(path); err == nil {
sc := bufio.NewScanner(f)
for sc.Scan() {
fields := strings.Fields(sc.Text())
if len(fields) != 3 || fields[0] != hostname {
continue
}
if fields[1] == key.Type() && fields[2] == fp {
f.Close()
return nil
}
if fields[1] == key.Type() {
f.Close()
return fmt.Errorf("ключ хоста %s изменился: был %s, теперь %s. Если так и должно быть, удалите строку с этим хостом из %s на агенте",
hostname, fields[2], fp, path)
}
}
f.Close()
}
_ = os.MkdirAll(filepath.Dir(path), 0o700)
f, err := os.OpenFile(path, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0o600)
if err == nil {
_, _ = f.WriteString(line + fp + "\n")
f.Close()
}
return nil
}
}
// sshAuth is how to sign in: the key given, the password given (also
// answered to keyboard-interactive prompts), or the agent user's own keys.
func sshAuth(t proto.SSHTarget) ([]ssh.AuthMethod, error) {
var methods []ssh.AuthMethod
if t.Key != "" {
var signer ssh.Signer
var err error
if t.Passphrase != "" {
signer, err = ssh.ParsePrivateKeyWithPassphrase([]byte(t.Key), []byte(t.Passphrase))
} else {
signer, err = ssh.ParsePrivateKey([]byte(t.Key))
}
var missing *ssh.PassphraseMissingError
if errors.As(err, &missing) {
return nil, errors.New("ключ защищён фразой - введите её")
}
if err != nil {
return nil, fmt.Errorf("ключ не читается: %v", err)
}
methods = append(methods, ssh.PublicKeys(signer))
}
if t.Password != "" {
pw := t.Password
methods = append(methods, ssh.Password(pw),
ssh.KeyboardInteractive(func(_, _ string, qs []string, _ []bool) ([]string, error) {
ans := make([]string, len(qs))
for i := range ans {
ans[i] = pw
}
return ans, nil
}))
}
if len(methods) == 0 {
var signers []ssh.Signer
for _, n := range []string{"id_ed25519", "id_ecdsa", "id_rsa"} {
b, err := os.ReadFile(filepath.Join(homeDir(""), ".ssh", n))
if err != nil {
continue
}
if s, err := ssh.ParsePrivateKey(b); err == nil {
signers = append(signers, s)
}
}
if len(signers) == 0 {
return nil, errors.New("не задан ни пароль, ни ключ, и в ~/.ssh агента нет ключей без фразы")
}
methods = append(methods, ssh.PublicKeys(signers...))
}
return methods, nil
}
// sshShell is the remote shell as a stream: its output (stdout and stderr,
// one on a PTY) to read, keystrokes to write.
type sshShell struct {
client *ssh.Client
sess *ssh.Session
stdin io.WriteCloser
out *io.PipeReader
once sync.Once
}
func (s *sshShell) Read(b []byte) (int, error) { return s.out.Read(b) }
func (s *sshShell) Write(b []byte) (int, error) { return s.stdin.Write(b) }
func (s *sshShell) Resize(cols, rows uint16) {
if cols > 0 && rows > 0 {
_ = s.sess.WindowChange(int(rows), int(cols))
}
}
func (s *sshShell) Close() error {
s.once.Do(func() {
_ = s.sess.Close()
_ = s.client.Close()
_ = s.out.Close()
})
return nil
}
// sshConnect connects and signs in to t, saying how it goes by status (the
// SSHStatus steps). It returns the client and the host key's fingerprint.
func (a *Agent) sshConnect(ctx context.Context, t proto.SSHTarget, status func(st, note string)) (*ssh.Client, string, error) {
if t.Port == 0 {
t.Port = 22
}
addr := net.JoinHostPort(t.Host, strconv.Itoa(t.Port))
auth, err := sshAuth(t)
if err != nil {
return nil, "", err
}
status(proto.SSHDial, "подключение к "+addr)
d := net.Dialer{Timeout: 15 * time.Second, KeepAlive: 30 * time.Second}
conn, err := d.DialContext(ctx, "tcp", addr)
if err != nil {
return nil, "", fmt.Errorf("не подключиться к %s: %v", addr, err)
}
status(proto.SSHAuth, "вход как "+t.User)
var fp string
_ = conn.SetDeadline(time.Now().Add(30 * time.Second))
c, chans, reqs, err := ssh.NewClientConn(conn, addr, &ssh.ClientConfig{
User: t.User,
Auth: auth,
HostKeyCallback: a.hostKeyCheck(&fp),
Timeout: 15 * time.Second,
})
if err != nil {
conn.Close()
msg := strings.TrimPrefix(err.Error(), "ssh: handshake failed: ")
if strings.Contains(msg, "unable to authenticate") {
msg = "сервер не принял вход как " + t.User + " - проверьте пароль или ключ"
}
return nil, "", errors.New(msg)
}
_ = conn.SetDeadline(time.Time{})
return ssh.NewClient(c, chans, reqs), fp, nil
}
// runSSH connects to the target and bridges its shell to the session.
func (a *Agent) runSSH(parent context.Context, sid string, t proto.SSHTarget) {
status := func(st, note string) {
a.postOut(context.Background(), sid, proto.OutMessage{Status: st, Note: note})
}
fail := func(err error) {
log.Printf("ssh %s: %v", sid, err)
status(proto.SSHErr, err.Error())
a.postOut(context.Background(), sid, proto.OutMessage{Data: b64.EncodeToString([]byte("\x1b[31m" + err.Error() + "\x1b[0m\r\n"))})
a.postOut(context.Background(), sid, proto.OutMessage{Exit: true})
}
client, fp, err := a.sshConnect(parent, t, status)
if err != nil {
fail(err)
return
}
port := t.Port
if port == 0 {
port = 22
}
addr := net.JoinHostPort(t.Host, strconv.Itoa(port))
sess, err := client.NewSession()
if err != nil {
client.Close()
fail(fmt.Errorf("сессия не открылась: %v", err))
return
}
modes := ssh.TerminalModes{ssh.ECHO: 1, ssh.TTY_OP_ISPEED: 115200, ssh.TTY_OP_OSPEED: 115200}
if err := sess.RequestPty("xterm-256color", 24, 80, modes); err != nil {
sess.Close()
client.Close()
fail(fmt.Errorf("терминал не выделен: %v", err))
return
}
stdin, _ := sess.StdinPipe()
pr, pw := io.Pipe()
sess.Stdout = pw
sess.Stderr = pw
if err := sess.Shell(); err != nil {
sess.Close()
client.Close()
fail(fmt.Errorf("оболочка не запустилась: %v", err))
return
}
sh := &sshShell{client: client, sess: sess, stdin: stdin, out: pr}
go func() { _ = sess.Wait(); pw.Close() }()
// Keep-alives, so a dead host is noticed and the session ends.
go func() {
tk := time.NewTicker(30 * time.Second)
defer tk.Stop()
for range tk.C {
if _, _, err := client.SendRequest("keepalive@openssh.com", true, nil); err != nil {
sh.Close()
return
}
}
}()
log.Printf("ssh %s: %s@%s up (host key %s)", sid, t.User, addr, fp)
status(proto.SSHOK, t.User+"@"+addr)
a.bridgeStream(parent, sid, sh, sh.Resize)
log.Printf("ssh %s: ended", sid)
}
+794
View File
@@ -0,0 +1,794 @@
package agent
// Files on an SSH host, for the panel's file panes: the agent keeps one SSH
// connection per pane and runs plain commands there - stat, cat, mv, rm - as
// the tar transfer does. No SFTP is needed on the host. A connection that
// dropped is opened again once with what it was opened with; one left alone
// for sshfsIdle is closed.
import (
"archive/tar"
"bytes"
"context"
"crypto/rand"
"encoding/base64"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"path"
"strconv"
"strings"
"sync"
"sync/atomic"
"time"
"unicode/utf8"
"golang.org/x/crypto/ssh"
"github.com/x-grk/longpolling-remote-access/internal/proto"
)
const sshfsIdle = 10 * time.Minute
type sshfsConn struct {
mu sync.Mutex
client *ssh.Client
target proto.SSHTarget // kept in the agent's memory to reconnect once after a drop
statFmt string // "gnu", "bsd" or "ls"
home string
last time.Time
busy int // transfers under way: the sweep leaves the connection be
}
// hold marks the connection busy for a long transfer; the returned func ends
// it and starts the idle clock again.
func (c *sshfsConn) hold() func() {
c.mu.Lock()
c.busy++
c.mu.Unlock()
return func() {
c.mu.Lock()
c.busy--
c.last = time.Now()
c.mu.Unlock()
}
}
// idle: nothing under way and unused for sshfsIdle by now.
func (c *sshfsConn) idle(now time.Time) bool {
c.mu.Lock()
defer c.mu.Unlock()
return c.busy == 0 && now.Sub(c.last) > sshfsIdle
}
var (
sshfsConns sync.Map // id -> *sshfsConn
sshfsJanitor sync.Once
)
// statCommand lists dir's entries as "type|size|mtime|mode|name" lines.
func statCommand(flavour, dir string) string {
d := shq(strings.TrimSuffix(dir, "/") + "/")
// Each line starts with LD for a link to a folder, so it opens like one.
loop := "cd " + d + " && for f in .[!.]* ..?* *; do [ -e \"$f\" ] || [ -L \"$f\" ] || continue; " +
"t=; [ -L \"$f\" ] && [ -d \"$f\" ] && t=LD; printf '%s|' \"$t\"; "
switch flavour {
case "bsd":
return loop + "stat -f '%HT|%z|%m|%Sp|%N' -- \"$f\"; done"
case "ls":
return "cd " + d + " && LC_ALL=C ls -la"
}
return loop + "stat -c '%F|%s|%Y|%A|%n' -- \"$f\"; done"
}
// parseStatLine reads one line of statCommand's output.
func parseStatLine(line string) (proto.FSEntry, bool) {
// The LD mark of statCommand: empty, or LD for a link to a folder.
mark := ""
if i := strings.IndexByte(line, '|'); i >= 0 && (line[:i] == "" || line[:i] == "LD") {
mark, line = line[:i], line[i+1:]
}
p := strings.SplitN(line, "|", 5)
if len(p) != 5 {
return proto.FSEntry{}, false
}
size, err1 := strconv.ParseInt(p[1], 10, 64)
mt, err2 := strconv.ParseInt(p[2], 10, 64)
if err1 != nil || err2 != nil {
return proto.FSEntry{}, false
}
t := strings.ToLower(p[0])
return proto.FSEntry{Name: p[4], IsDir: strings.Contains(t, "directory") || mark == "LD", Size: size, MTime: mt, Mode: p[3],
Link: strings.Contains(t, "link")}, true
}
// parseLsLine reads a line of "ls -la" for a host with no stat: no times.
func parseLsLine(line string) (proto.FSEntry, bool) {
f := strings.Fields(line)
if len(f) < 9 || len(f[0]) < 10 {
return proto.FSEntry{}, false
}
size, err := strconv.ParseInt(f[4], 10, 64)
if err != nil {
return proto.FSEntry{}, false
}
// The name is what follows the eighth field, spaces and all.
rest := line
for i := 0; i < 8; i++ {
rest = strings.TrimLeft(rest, " ")
rest = rest[strings.IndexByte(rest, ' ')+1:]
}
name := strings.TrimLeft(rest, " ")
if f[0][0] == 'l' {
if i := strings.Index(name, " -> "); i >= 0 {
name = name[:i]
}
}
if name == "." || name == ".." {
return proto.FSEntry{}, false
}
return proto.FSEntry{Name: name, IsDir: f[0][0] == 'd', Size: size, Mode: f[0]}, true
}
func randHex(n int) string {
b := make([]byte, n)
_, _ = rand.Read(b)
return hex.EncodeToString(b)
}
// sshfsOpen connects for a file pane and tells the host's home.
func (a *Agent) sshfsOpen(ctx context.Context, t proto.SSHTarget) (any, error) {
client, _, err := a.sshConnect(ctx, t, func(string, string) {})
if err != nil {
return nil, err
}
c := &sshfsConn{client: client, target: t, last: time.Now()}
out, _ := sshRun(client, "stat -c %F / >/dev/null 2>&1 && echo GNU; stat -f %HT / >/dev/null 2>&1 && echo BSD; "+
"command -v zip >/dev/null 2>&1 && echo ZIP; command -v unzip >/dev/null 2>&1 && echo UNZIP; "+
"command -v python3 >/dev/null 2>&1 && echo PY; echo \"$HOME\"")
lines := strings.Split(strings.TrimSpace(out), "\n")
switch {
case strings.Contains(out, "GNU"):
c.statFmt = "gnu"
case strings.Contains(out, "BSD"):
c.statFmt = "bsd"
default:
c.statFmt = "ls"
}
c.home = strings.TrimSpace(lines[len(lines)-1])
if !strings.HasPrefix(c.home, "/") {
c.home = "/"
}
id := randHex(12)
sshfsConns.Store(id, c)
sshfsJanitor.Do(func() { go sshfsSweep() })
// zip and unzip, where the host has them, for the pane's buttons.
return map[string]any{"id": id, "home": c.home,
"zip": hasLine(lines, "ZIP") || hasLine(lines, "PY"), "unzip": hasLine(lines, "UNZIP") || hasLine(lines, "PY")}, nil
}
func hasLine(lines []string, w string) bool {
for _, l := range lines {
if strings.TrimSpace(l) == w {
return true
}
}
return false
}
// sshfsSweep closes connections nobody used for sshfsIdle.
func sshfsSweep() {
for range time.Tick(time.Minute) {
sshfsConns.Range(func(k, v any) bool {
if v.(*sshfsConn).idle(time.Now()) {
sshfsClose(k.(string))
}
return true
})
}
}
func sshfsClose(id string) {
if v, ok := sshfsConns.LoadAndDelete(id); ok {
_ = v.(*sshfsConn).client.Close()
}
}
func sshfsGet(id string) (*sshfsConn, error) {
v, ok := sshfsConns.Load(id)
if !ok {
return nil, errors.New("подключение закрыто - откройте файлы по SSH снова")
}
c := v.(*sshfsConn)
c.mu.Lock()
c.last = time.Now()
c.mu.Unlock()
return c, nil
}
// session opens an SSH session on the connection; a connection that dropped
// is opened again once.
func (a *Agent) sshfsSession(c *sshfsConn) (*ssh.Session, error) {
c.mu.Lock()
client := c.client
c.mu.Unlock()
s, err := client.NewSession()
if err == nil {
return s, nil
}
ctx, cancel := context.WithTimeout(context.Background(), 40*time.Second)
defer cancel()
nc, _, cerr := a.sshConnect(ctx, c.target, func(string, string) {})
if cerr != nil {
return nil, fmt.Errorf("подключение к хосту потеряно: %v", cerr)
}
c.mu.Lock()
_ = c.client.Close()
c.client = nc
c.mu.Unlock()
return nc.NewSession()
}
// run runs cmd on the host with stdin, returning its output; a failure
// carries what the command said on stderr.
func (a *Agent) sshfsRun(c *sshfsConn, cmd string, stdin io.Reader) ([]byte, error) {
s, err := a.sshfsSession(c)
if err != nil {
return nil, err
}
defer s.Close()
var out, errb bytes.Buffer
s.Stdout = &out
s.Stderr = &limitBuf{b: &errb, max: 4096}
if stdin != nil {
s.Stdin = stdin
}
if err := s.Run(posix(cmd)); err != nil {
msg := strings.TrimSpace(errb.String())
if msg == "" {
msg = err.Error()
}
if len(msg) > 300 {
msg = msg[:300]
}
return out.Bytes(), errors.New(msg)
}
return out.Bytes(), nil
}
// sshfsTask runs one sshfs.* task.
func (a *Agent) sshfsTask(ctx context.Context, env proto.TaskEnvelope) (any, error) {
var args proto.SSHFSArgs
if err := json.Unmarshal(env.Args, &args); err != nil {
return nil, err
}
switch env.Type {
case proto.TaskSSHFSOpen:
if args.SSH == nil {
return nil, errors.New("не указан хост")
}
return a.sshfsOpen(ctx, *args.SSH)
case proto.TaskSSHFSClose:
sshfsClose(args.Conn)
return nil, nil
}
c, err := sshfsGet(args.Conn)
if err != nil {
return nil, err
}
return a.sshfsOp(c, args.Op, args.Args)
}
func (a *Agent) sshfsOp(c *sshfsConn, op string, raw json.RawMessage) (any, error) {
switch op {
case "list":
var p proto.FSPathArgs
_ = json.Unmarshal(raw, &p)
dir := p.Path
if dir == "" || strings.HasPrefix(dir, "~") {
dir = c.home
}
out, err := a.sshfsRun(c, statCommand(c.statFmt, dir), nil)
if err != nil && len(out) == 0 {
return nil, err
}
entries := []proto.FSEntry{}
for _, line := range strings.Split(string(out), "\n") {
var e proto.FSEntry
var ok bool
if c.statFmt == "ls" {
e, ok = parseLsLine(line)
} else {
e, ok = parseStatLine(line)
}
if ok {
entries = append(entries, e)
}
}
return proto.FSListResult{Path: path.Clean(dir), Entries: entries}, nil
case "read":
var p proto.FSPathArgs
_ = json.Unmarshal(raw, &p)
size, _ := a.sshfsSize(c, p.Path)
out, err := a.sshfsRun(c, "head -c "+strconv.Itoa(proto.FSReadCap+1)+" -- "+shq(p.Path), nil)
if err != nil {
return nil, err
}
truncated := len(out) > proto.FSReadCap
if truncated {
out = out[:proto.FSReadCap]
}
return proto.FSReadResult{Path: p.Path, Size: size, Content: base64.StdEncoding.EncodeToString(out),
Truncated: truncated, Binary: !utf8.Valid(out) || bytes.IndexByte(out, 0) >= 0}, nil
case "write":
var p proto.FSWriteArgs
_ = json.Unmarshal(raw, &p)
data, err := base64.StdEncoding.DecodeString(p.Data)
if err != nil {
return nil, err
}
_, err = a.sshfsRun(c, writeCommand(p.Path), bytes.NewReader(data))
return map[string]string{"path": p.Path}, err
case "mkdir":
var p proto.FSPathArgs
_ = json.Unmarshal(raw, &p)
_, err := a.sshfsRun(c, "mkdir -p -- "+shq(p.Path), nil)
return map[string]string{"path": p.Path}, err
case "delete", "deltree":
var p proto.FSPathArgs
_ = json.Unmarshal(raw, &p)
if p.Path == "" || path.Clean(p.Path) == "/" {
return nil, errors.New("отказ: удаление корня запрещено")
}
flag := "-f"
if op == "deltree" {
flag = "-rf"
}
_, err := a.sshfsRun(c, "rm "+flag+" -- "+shq(p.Path), nil)
return map[string]string{"path": p.Path}, err
case "move":
var p proto.FSMoveArgs
_ = json.Unmarshal(raw, &p)
_, err := a.sshfsRun(c, "mv -- "+shq(p.Src)+" "+shq(p.Dst), nil)
return map[string]string{"src": p.Src, "dst": p.Dst}, err
case "copy":
var p proto.FSCopyArgs
_ = json.Unmarshal(raw, &p)
return nil, a.sshfsCopy(c, p)
case "zip":
var p proto.FSZipArgs
_ = json.Unmarshal(raw, &p)
cmd, err := zipCommand(p.Paths, p.Dest)
if err != nil {
return nil, err
}
_, err = a.sshfsRun(c, cmd, nil)
return map[string]string{"dest": p.Dest}, err
case "unzip":
var p proto.FSUnzipArgs
_ = json.Unmarshal(raw, &p)
_, err := a.sshfsRun(c, unzipCommand(p.Path, p.Dest), nil)
return map[string]string{"dest": p.Dest}, err
case "stat":
var p proto.FSPathArgs
_ = json.Unmarshal(raw, &p)
return a.sshfsStat(c, p.Path)
case "chmod":
var p proto.FSChmodArgs
_ = json.Unmarshal(raw, &p)
rec := ""
if p.Recursive {
rec = "-R "
}
var cmds []string
if p.Mode != nil {
cmds = append(cmds, "chmod "+rec+fmt.Sprintf("%o", *p.Mode)+" -- "+shq(p.Path))
}
if p.Owner != "" || p.Group != "" {
who := p.Owner
if p.Group != "" {
who += ":" + p.Group
}
cmds = append(cmds, "chown "+rec+shq(who)+" -- "+shq(p.Path))
}
if len(cmds) == 0 {
return nil, nil
}
_, err := a.sshfsRun(c, strings.Join(cmds, " && "), nil)
return map[string]string{"path": p.Path}, err
}
return nil, errors.New("неизвестная операция")
}
// zipCommand packs paths - items of one folder - into the archive dest, from
// that folder, under their own names. Paths are absolute and names start
// with "./", so none reads as an option.
func zipCommand(paths []string, dest string) (string, error) {
if len(paths) == 0 {
return "", errors.New("не выбраны файлы")
}
dir := path.Dir(path.Clean(paths[0]))
names := make([]string, len(paths))
for i, p := range paths {
p = path.Clean(p)
if path.Dir(p) != dir {
return "", errors.New("в архив - только из одной папки")
}
names[i] = shq("./" + path.Base(p))
}
// zip where the host has it, else Python's zipfile, which most have.
items := strings.Join(names, " ")
return "cd -- " + shq(dir) + " && if command -v zip >/dev/null 2>&1; then zip -r -q -y " + shq(path.Clean(dest)) + " " + items + "; " +
"else python3 -m zipfile -c " + shq(path.Clean(dest)) + " " + items + "; fi", nil
}
// unzipCommand unpacks the archive p into dest (its own folder if empty),
// replacing what is there, as on a machine.
func unzipCommand(p, dest string) string {
if dest == "" {
dest = path.Dir(path.Clean(p))
}
return "if command -v unzip >/dev/null 2>&1; then unzip -o -q " + shq(path.Clean(p)) + " -d " + shq(path.Clean(dest)) + "; " +
"else python3 -m zipfile -e " + shq(path.Clean(p)) + " " + shq(path.Clean(dest)) + "; fi"
}
// writeCommand writes stdin to p through a file beside it, so a broken
// transfer leaves the old file as it was.
func writeCommand(p string) string {
tmp := path.Join(path.Dir(p), "."+path.Base(p)+".lpra-"+randHex(4))
return "cat > " + shq(tmp) + " && mv -f -- " + shq(tmp) + " " + shq(p) + " || { rm -f -- " + shq(tmp) + "; exit 1; }"
}
// sizeCommand tells the size of p - of its target, if a link: what cat
// reads.
func sizeCommand(statFmt, p string) string {
switch statFmt {
case "bsd":
return "stat -L -f %z -- " + shq(p)
case "ls":
return "wc -c < " + shq(p)
}
return "stat -L -c %s -- " + shq(p)
}
func (a *Agent) sshfsSize(c *sshfsConn, p string) (int64, error) {
out, err := a.sshfsRun(c, sizeCommand(c.statFmt, p), nil)
if err != nil {
return 0, err
}
return strconv.ParseInt(strings.TrimSpace(string(out)), 10, 64)
}
func (a *Agent) sshfsStat(c *sshfsConn, p string) (any, error) {
cmd := "stat -c '%F|%s|%Y|%a|%u|%g|%U|%G' -- " + shq(p)
if c.statFmt == "bsd" {
cmd = "stat -f '%HT|%z|%m|%Lp|%u|%g|%Su|%Sg' -- " + shq(p)
}
out, err := a.sshfsRun(c, cmd, nil)
if err != nil {
return nil, err
}
f := strings.Split(strings.TrimSpace(string(out)), "|")
if len(f) != 8 {
return nil, errors.New("не удалось прочитать свойства")
}
size, _ := strconv.ParseInt(f[1], 10, 64)
mt, _ := strconv.ParseInt(f[2], 10, 64)
mode, _ := strconv.ParseUint(f[3], 8, 32)
uid, _ := strconv.Atoi(f[4])
gid, _ := strconv.Atoi(f[5])
return proto.FSStatResult{Path: p, IsDir: strings.Contains(strings.ToLower(f[0]), "directory"), Size: size, MTime: mt,
Mode: uint32(mode), UID: uid, GID: gid, Owner: f[6], Group: f[7], Users: []string{}, Groups: []string{}}, nil
}
// sshfsCopy copies or moves on the host, by the conflict rule for what is
// there already.
func (a *Agent) sshfsCopy(c *sshfsConn, p proto.FSCopyArgs) error {
src, dst := shq(p.Src), shq(p.Dst)
exists, _ := a.sshfsRun(c, "test -e "+dst+" -o -L "+dst+" && echo EXISTS", nil)
there := strings.Contains(string(exists), "EXISTS")
if there {
switch p.Conflict {
case "replace":
if _, err := a.sshfsRun(c, "rm -rf -- "+dst, nil); err != nil {
return err
}
case "skip":
return nil
default:
return fmt.Errorf("%s уже существует", p.Dst)
}
}
cmd := "cp -a -- " + src + " " + dst
if p.Move {
cmd = "mv -- " + src + " " + dst
}
_, err := a.sshfsRun(c, cmd, nil)
return err
}
// doSSHFSSend sends a file of the host to the browser in pieces, as
// doFSSend does with a local one: cat's output, piece after piece.
func (a *Agent) doSSHFSSend(ctx context.Context, env proto.TaskEnvelope) proto.TaskResult {
var args proto.FSXferArgs
if err := json.Unmarshal(env.Args, &args); err != nil || args.Chunk <= 0 {
return proto.TaskResult{Error: "плохой запрос"}
}
c, err := sshfsGet(args.Conn)
if err != nil {
return proto.TaskResult{Error: err.Error()}
}
defer c.hold()()
size, err := a.sshfsSize(c, args.Path)
if err != nil {
return proto.TaskResult{Error: err.Error()}
}
if size > proto.FSFileCap {
return proto.TaskResult{Error: "файл больше 512 МБ"}
}
s, err := a.sshfsSession(c)
if err != nil {
return proto.TaskResult{Error: err.Error()}
}
defer s.Close()
out, _ := s.StdoutPipe()
if err := s.Start(posix("cat -- " + shq(args.Path))); err != nil {
return proto.TaskResult{Error: err.Error()}
}
// Read in order from cat, posted to the server a few at a time.
sizeHdr := strconv.FormatInt(size, 10)
n := pieceCount(size, args.Chunk)
pctx, cancel := context.WithCancel(ctx)
defer cancel()
var (
wg sync.WaitGroup
once sync.Once
first error
)
fail := func(err error) { once.Do(func() { first = err; cancel() }) }
sem := make(chan struct{}, xferWorkers)
for i := 0; i < n && pctx.Err() == nil; i++ {
buf := make([]byte, min(int64(args.Chunk), size-int64(i)*int64(args.Chunk)))
if _, err := io.ReadFull(out, buf); err != nil {
fail(errors.New("файл изменился во время чтения: " + err.Error()))
break
}
select {
case sem <- struct{}{}:
case <-pctx.Done():
}
if pctx.Err() != nil {
break
}
wg.Add(1)
go func(i int, buf []byte) {
defer func() { <-sem; wg.Done() }()
err := a.xferRequest(pctx, func(ctx context.Context) (*http.Request, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, a.xferURL(args.Xfer, i), bytes.NewReader(buf))
if err == nil {
req.Header.Set("X-Size", sizeHdr)
req.Header.Set("Content-Type", "application/octet-stream")
}
return req, err
}, func(*http.Response) error { return nil })
if err != nil {
fail(errors.New("передача прервана: " + err.Error()))
}
}(i, buf)
}
wg.Wait()
if first == nil && ctx.Err() != nil {
first = ctx.Err()
}
if first != nil {
return proto.TaskResult{Error: first.Error()}
}
// All of it is read; the session closes (deferred) without waiting on a
// cat that may have more to say.
data, _ := json.Marshal(map[string]any{"path": args.Path, "size": size})
return proto.TaskResult{OK: true, Data: data}
}
// doSSHFSPut writes a file the browser uploads onto the host, through a file
// beside it.
func (a *Agent) doSSHFSPut(ctx context.Context, env proto.TaskEnvelope) proto.TaskResult {
var args proto.FSXferArgs
if err := json.Unmarshal(env.Args, &args); err != nil || args.Chunk <= 0 || args.Size < 0 || args.Size > proto.FSFileCap {
return proto.TaskResult{Error: "плохой запрос"}
}
c, err := sshfsGet(args.Conn)
if err != nil {
return proto.TaskResult{Error: err.Error()}
}
defer c.hold()()
s, err := a.sshfsSession(c)
if err != nil {
return proto.TaskResult{Error: err.Error()}
}
defer s.Close()
in, _ := s.StdinPipe()
var errb bytes.Buffer
s.Stderr = &limitBuf{b: &errb, max: 4096}
if err := s.Start(posix(writeCommand(args.Path))); err != nil {
return proto.TaskResult{Error: err.Error()}
}
// Pieces come in a few ahead; stdin takes them in order. ack: how many
// are written, for the server to let go.
n := pieceCount(args.Size, args.Chunk)
var written atomic.Int64
err = ordered(ctx, n, xferWorkers, func(ctx context.Context, i int) ([]byte, error) {
want := min(int64(args.Chunk), args.Size-int64(i)*int64(args.Chunk))
var data []byte
err := a.xferRequest(ctx, func(ctx context.Context) (*http.Request, error) {
return http.NewRequestWithContext(ctx, http.MethodGet, a.xferURL(args.Xfer, i)+"?ack="+strconv.FormatInt(written.Load(), 10), nil)
}, func(resp *http.Response) error {
b, err := io.ReadAll(io.LimitReader(resp.Body, int64(args.Chunk)+1))
if err == nil && int64(len(b)) != want {
err = errors.New("кусок не той длины")
}
data = b
return err
})
return data, err
}, func(i int, b []byte) error {
_, err := in.Write(b)
written.Store(int64(i + 1))
return err
})
if err != nil {
_ = s.Close()
return proto.TaskResult{Error: "загрузка прервана: " + err.Error()}
}
_ = in.Close()
if err := s.Wait(); err != nil {
msg := strings.TrimSpace(errb.String())
if msg == "" {
msg = err.Error()
}
return proto.TaskResult{Error: msg}
}
data, _ := json.Marshal(map[string]any{"path": args.Path, "size": args.Size})
return proto.TaskResult{OK: true, Data: data}
}
// renameTar copies a tar stream, giving each top entry named in names - and
// everything under it - its new name: what the host's tar packs under the
// file's own name arrives under the name chosen for it. Other entries (the
// "._" companions a macOS tar adds) are left out.
func renameTar(w io.Writer, r io.Reader, names map[string]string) error {
tr, tw := tar.NewReader(r), tar.NewWriter(w)
for {
h, err := tr.Next()
if err == io.EOF {
return tw.Close()
}
if err != nil {
return err
}
n := strings.TrimPrefix(h.Name, "./")
top, rest := n, ""
if i := strings.IndexByte(n, '/'); i >= 0 {
top, rest = n[:i], n[i:]
}
// Only base and what is under it - nothing that climbs out by "..".
name, ok := names[top]
if !ok || strings.Contains("/"+rest+"/", "/../") {
continue
}
h.Name = name + rest
h.Uid, h.Gid, h.Uname, h.Gname = 0, 0, "", ""
if err := tw.WriteHeader(h); err != nil {
return err
}
if _, err := io.Copy(tw, tr); err != nil {
return err
}
}
}
// packRemote writes src on the connection's host as a tar stream named name,
// counting what goes for the progress.
func (a *Agent) packRemote(ctx context.Context, w io.Writer, c *sshfsConn, srcs, names []string, m *shipMeter) error {
defer c.hold()()
s, err := a.sshfsSession(c)
if err != nil {
return err
}
defer s.Close()
go func() { <-ctx.Done(); s.Close() }()
out, _ := s.StdoutPipe()
var errb bytes.Buffer
s.Stderr = &limitBuf{b: &errb, max: 4096}
// All from one folder: packed from there by their own names.
dir := path.Dir(path.Clean(srcs[0]))
rename := map[string]string{}
cmd := "COPYFILE_DISABLE=1 tar -cf - -C " + shq(dir) + " --"
for i, p := range srcs {
b := path.Base(path.Clean(p))
rename[b] = names[i]
cmd += " " + shq(b)
}
if err := s.Start(posix(cmd)); err != nil {
return err
}
if err := renameTar(w, &countReader{r: out, n: &m.done}, rename); err != nil {
return err
}
if err := s.Wait(); err != nil {
if msg := strings.TrimSpace(errb.String()); msg != "" {
return errors.New("tar на хосте: " + msg)
}
return err
}
return nil
}
// remoteSize is about how many bytes src takes on the host, for progress.
func (a *Agent) remoteSize(c *sshfsConn, src string) int64 {
out, err := a.sshfsRun(c, "du -sk -- "+shq(src), nil)
if err != nil {
return 0
}
f := strings.Fields(string(out))
if len(f) == 0 {
return 0
}
kb, _ := strconv.ParseInt(f[0], 10, 64)
return kb * 1024
}
// unpackRemote puts a tar stream into dir on the connection's host, by the
// conflict rule: tar keeps or replaces what is there, an "error" rule
// refuses when name is there already.
func (a *Agent) unpackRemote(c *sshfsConn, r io.Reader, dir string, names []string, conflict string) (skipped int, err error) {
defer c.hold()()
base := strings.TrimSuffix(dir, "/") + "/"
check := ""
for _, n := range names {
check += "test -e " + shq(base+n) + " -o -L " + shq(base+n) + " && echo " + shq("LPRA_EXISTS "+n) + "; "
}
out, _ := a.sshfsRun(c, check+"true", nil)
keep := map[string]string{}
var put []string
for _, n := range names {
if strings.Contains(string(out), "LPRA_EXISTS "+n+"\n") {
switch conflict {
case "skip":
// Left as it was, as on a machine.
skipped++
continue
case "replace":
default:
return 0, errors.New(base + n + " уже существует")
}
}
keep[n] = n
put = append(put, n)
}
if len(put) == 0 {
return skipped, nil
}
// Only the items sent (and not kept back) reach the host's tar.
pr, pw := io.Pipe()
go func() { pw.CloseWithError(renameTar(pw, r, keep)) }()
defer pr.Close()
_, err = a.sshfsRun(c, unpackCommand(dir, put, conflict), pr)
return skipped, err
}
// unpackCommand unpacks a tar stream of names into dir; "replace" removes
// what is there first, so each thing is replaced whole as on a machine.
func unpackCommand(dir string, names []string, conflict string) string {
cmd := "mkdir -p -- " + shq(dir) + " && cd -- " + shq(dir) + " && tar -xf -"
if conflict == "replace" {
rm := "rm -rf --"
for _, n := range names {
rm += " " + shq(strings.TrimSuffix(dir, "/")+"/"+n)
}
cmd = rm + " && " + cmd
}
return cmd
}
+242
View File
@@ -0,0 +1,242 @@
package agent
import (
"archive/tar"
"bytes"
"io"
"os"
"path/filepath"
"strings"
"testing"
"time"
)
func TestParseStatLine(t *testing.T) {
// statCommand prints: type|size|mtime|mode|name
e, ok := parseStatLine("directory|4096|1760000000|drwxr-xr-x|my dir")
if !ok || !e.IsDir || e.Name != "my dir" || e.MTime != 1760000000 || e.Mode != "drwxr-xr-x" {
t.Fatalf("dir: %+v %v", e, ok)
}
e, ok = parseStatLine("regular file|12|1760000001|-rw-r--r--|a|b.txt")
if !ok || e.IsDir || e.Size != 12 || e.Name != "a|b.txt" {
t.Fatalf("file with | in name: %+v %v", e, ok)
}
if _, ok := parseStatLine("garbage"); ok {
t.Fatal("garbage parsed")
}
}
func TestStatCommandQuotes(t *testing.T) {
for _, fl := range []string{"gnu", "bsd"} {
c := statCommand(fl, "/srv/it's here")
if !strings.Contains(c, `'/srv/it'\''s here/'`) {
t.Errorf("%s: dir not quoted: %s", fl, c)
}
}
}
func TestParseLsLine(t *testing.T) {
e, ok := parseLsLine("-rw-r--r-- 1 root root 1234 Oct 15 10:00 my file.txt")
if !ok || e.Name != "my file.txt" || e.Size != 1234 || e.IsDir {
t.Fatalf("file: %+v %v", e, ok)
}
e, ok = parseLsLine("lrwxrwxrwx 1 root root 7 Oct 15 10:00 link -> target")
if !ok || e.Name != "link" {
t.Fatalf("link: %+v %v", e, ok)
}
if _, ok := parseLsLine("drwxr-xr-x 2 root root 4096 Oct 15 10:00 .."); ok {
t.Fatal(".. listed")
}
if _, ok := parseLsLine("total 8"); ok {
t.Fatal("total parsed")
}
}
func TestRenameTarEntries(t *testing.T) {
var src bytes.Buffer
tw := tar.NewWriter(&src)
for _, n := range []string{"old/", "old/a.txt", "old/sub/b"} {
typ := byte(tar.TypeReg)
if strings.HasSuffix(n, "/") {
typ = tar.TypeDir
}
_ = tw.WriteHeader(&tar.Header{Name: n, Mode: 0o644, Typeflag: typ})
}
_ = tw.Close()
var out bytes.Buffer
if err := renameTar(&out, &src, map[string]string{"old": "new"}); err != nil {
t.Fatal(err)
}
tr := tar.NewReader(&out)
var names []string
for {
h, err := tr.Next()
if err != nil {
break
}
names = append(names, h.Name)
}
if strings.Join(names, ",") != "new/,new/a.txt,new/sub/b" {
t.Fatalf("names: %v", names)
}
}
// A macOS host's tar packs "._name" companions with the file's attributes;
// they are not the file and must not land under its name.
func TestRenameTarDropsOtherEntries(t *testing.T) {
var src bytes.Buffer
tw := tar.NewWriter(&src)
for _, e := range []struct{ n, body string }{{"./._r.txt", "attrs"}, {"./r.txt", "data"}} {
_ = tw.WriteHeader(&tar.Header{Name: e.n, Mode: 0o644, Typeflag: tar.TypeReg, Size: int64(len(e.body))})
_, _ = tw.Write([]byte(e.body))
}
_ = tw.Close()
var out bytes.Buffer
if err := renameTar(&out, &src, map[string]string{"r.txt": "new.txt"}); err != nil {
t.Fatal(err)
}
tr := tar.NewReader(&out)
var got []string
for {
h, err := tr.Next()
if err != nil {
break
}
b, _ := io.ReadAll(tr)
got = append(got, h.Name+"="+string(b))
}
if strings.Join(got, ",") != "new.txt=data" {
t.Fatalf("entries: %v", got)
}
}
// zip on an SSH host: from the items' folder, by their own names, quoted.
func TestZipCommand(t *testing.T) {
cmd, err := zipCommand([]string{"/srv/a b", "/srv/it's"}, "/tmp/x.zip")
if err != nil {
t.Fatal(err)
}
want := `cd -- '/srv' && if command -v zip >/dev/null 2>&1; then zip -r -q -y '/tmp/x.zip' './a b' './it'\''s'; ` +
`else python3 -m zipfile -c '/tmp/x.zip' './a b' './it'\''s'; fi`
if cmd != want {
t.Fatalf("got %s\nwant %s", cmd, want)
}
if _, err := zipCommand([]string{"/a/x", "/b/y"}, "/t.zip"); err == nil {
t.Fatal("items of two folders: want an error")
}
if _, err := zipCommand(nil, "/t.zip"); err == nil {
t.Fatal("no items: want an error")
}
if got := unzipCommand("/srv/a.zip", ""); got != `if command -v unzip >/dev/null 2>&1; then unzip -o -q '/srv/a.zip' -d '/srv'; else python3 -m zipfile -e '/srv/a.zip' '/srv'; fi` {
t.Fatalf("unzip: %s", got)
}
}
// A connection busy with a long transfer is not idle, however long ago it
// was opened; once the transfer ends the idle clock starts again.
func TestSweepSparesBusyConn(t *testing.T) {
c := &sshfsConn{last: time.Now().Add(-2 * sshfsIdle)}
release := c.hold()
if c.idle(time.Now()) {
t.Fatal("a busy connection was taken for idle")
}
release()
if c.idle(time.Now()) {
t.Fatal("just released: not idle yet")
}
if !c.idle(time.Now().Add(2 * sshfsIdle)) {
t.Fatal("unused for long: idle")
}
}
// A file's size is its target's when it is a link: what cat will read.
func TestSizeCommandFollowsLinks(t *testing.T) {
for _, f := range []string{"gnu", "bsd"} {
if cmd := sizeCommand(f, "/x"); !strings.Contains(cmd, "stat -L ") {
t.Fatalf("%s: %s", f, cmd)
}
}
}
// Nothing of a tar stream lands outside name: other entries and ".." paths
// are dropped.
func TestRenameTarKeepsInsideName(t *testing.T) {
var src bytes.Buffer
tw := tar.NewWriter(&src)
for _, n := range []string{"d/a", "d/../../etc/x", ".ssh/authorized_keys", "d/b"} {
_ = tw.WriteHeader(&tar.Header{Name: n, Mode: 0o644, Typeflag: tar.TypeReg})
}
_ = tw.Close()
var out bytes.Buffer
if err := renameTar(&out, &src, map[string]string{"d": "d"}); err != nil {
t.Fatal(err)
}
tr := tar.NewReader(&out)
var got []string
for {
h, err := tr.Next()
if err != nil {
break
}
got = append(got, h.Name)
}
if strings.Join(got, ",") != "d/a,d/b" {
t.Fatalf("entries: %v", got)
}
}
// Into an SSH host, "replace" replaces the thing whole, as on a machine.
func TestUnpackCommandReplaces(t *testing.T) {
cmd := unpackCommand("/srv", []string{"d"}, "replace")
if !strings.HasPrefix(cmd, "rm -rf -- '/srv/d' && ") || strings.Contains(cmd, "--overwrite") {
t.Fatalf("replace: %s", cmd)
}
if cmd := unpackCommand("/srv", []string{"d"}, "error"); cmd != "mkdir -p -- '/srv' && cd -- '/srv' && tar -xf -" {
t.Fatalf("error: %s", cmd)
}
}
// A link to a folder lists as a folder (marked as a link), so it opens.
func TestParseStatLineFolderLink(t *testing.T) {
e, ok := parseStatLine("LD|symbolic link|7|1700000000|lrwxrwxrwx|www")
if !ok || !e.IsDir || !e.Link || e.Name != "www" {
t.Fatalf("%+v %v", e, ok)
}
e, ok = parseStatLine("|regular file|5|1700000000|-rw-r--r--|a|b")
if !ok || e.IsDir || e.Name != "a|b" {
t.Fatalf("plain: %+v %v", e, ok)
}
if !strings.Contains(statCommand("gnu", "/x"), `[ -L "$f" ] && [ -d "$f" ]`) {
t.Fatal("the command does not mark folder links")
}
}
func TestListFolderLink(t *testing.T) {
d := t.TempDir()
_ = os.Mkdir(filepath.Join(d, "real"), 0o755)
_ = os.Symlink(filepath.Join(d, "real"), filepath.Join(d, "ln"))
r, err := fsList(d)
if err != nil {
t.Fatal(err)
}
for _, e := range r.Entries {
if e.Name == "ln" && (!e.IsDir || !e.Link) {
t.Fatalf("link: %+v", e)
}
}
}
// Commands go to the host as one argument of sh -c, so the user's own login
// shell (fish, csh) cannot read them its way.
func TestPosixCommand(t *testing.T) {
if got := posix(`if [ -d '/x' ]; then echo ok; fi`); got != `sh -c 'if [ -d '\''/x'\'' ]; then echo ok; fi'` {
t.Fatalf("%s", got)
}
}
// csh expands "!" even inside single quotes: it goes out as '\!'.
func TestPosixCommandBang(t *testing.T) {
if got := posix(`ls .[!.]*`); got != `sh -c 'ls .['\!'.]*'` {
t.Fatalf("%s", got)
}
}
+45
View File
@@ -0,0 +1,45 @@
package agent
import (
"runtime"
"sync"
"syscall"
"github.com/x-grk/longpolling-remote-access/internal/proto"
)
// statMu serializes samples: the CPU figure is a delta against the last one.
var statMu sync.Mutex
// sysStat samples the machine's load. It costs nothing while nobody asks: the
// panel requests it only for an open status page.
func sysStat() (proto.SysStatResult, error) {
statMu.Lock()
defer statMu.Unlock()
res := proto.SysStatResult{NCPU: runtime.NumCPU()}
cpu, err := cpuPercent()
if err != nil {
return res, err
}
res.CPU = cpu
if res.MemTotal, res.MemUsed, err = memUsage(); err != nil {
return res, err
}
var fs syscall.Statfs_t
if err := syscall.Statfs("/", &fs); err != nil {
return res, err
}
res.DiskTotal = uint64(fs.Blocks) * uint64(fs.Bsize)
res.DiskFree = uint64(fs.Bavail) * uint64(fs.Bsize)
return res, nil
}
func clampPercent(v float64) float64 {
if v < 0 {
return 0
}
if v > 100 {
return 100
}
return v
}
+43
View File
@@ -0,0 +1,43 @@
package agent
import (
"bytes"
"os/exec"
"runtime"
"strconv"
"golang.org/x/sys/unix"
)
// cpuPercent sums the per-process CPU shares ps reports. macOS exposes the
// host CPU ticks only through Mach calls, which need cgo.
func cpuPercent() (float64, error) {
out, err := exec.Command("/bin/ps", "-A", "-o", "%cpu=").Output()
if err != nil {
return 0, err
}
var sum float64
for _, f := range bytes.Fields(out) {
// Some locales print a decimal comma.
v, _ := strconv.ParseFloat(string(bytes.Replace(f, []byte(","), []byte("."), 1)), 64)
sum += v
}
return clampPercent(sum / float64(runtime.NumCPU())), nil
}
// memUsage takes the kernel's own share of memory still available
// (kern.memorystatus_level, what `memory_pressure` prints).
func memUsage() (total, used uint64, err error) {
total, err = unix.SysctlUint64("hw.memsize")
if err != nil {
return 0, 0, err
}
level, err := unix.SysctlUint32("kern.memorystatus_level")
if err != nil {
return 0, 0, err
}
if level > 100 {
level = 100
}
return total, total / 100 * uint64(100-level), nil
}
+121
View File
@@ -0,0 +1,121 @@
package agent
import (
"bytes"
"errors"
"os"
"strconv"
"time"
)
// Previous /proc/stat reading; the load is the busy share of the ticks since.
var (
cpuPrevIdle, cpuPrevTotal uint64
cpuPrevAt time.Time
)
// cpuStaleAfter is how old the previous reading may be: past it the delta
// would average over a stretch nobody was watching, so take a fresh pair.
const cpuStaleAfter = 15 * time.Second
func cpuPercent() (float64, error) {
if time.Since(cpuPrevAt) > cpuStaleAfter {
idle, total, err := readCPUTicks()
if err != nil {
return 0, err
}
cpuPrevIdle, cpuPrevTotal = idle, total
time.Sleep(200 * time.Millisecond)
}
idle, total, err := readCPUTicks()
if err != nil {
return 0, err
}
dIdle, dTotal := idle-cpuPrevIdle, total-cpuPrevTotal
cpuPrevIdle, cpuPrevTotal, cpuPrevAt = idle, total, time.Now()
if dTotal == 0 || dIdle > dTotal {
return 0, nil
}
return clampPercent(100 * float64(dTotal-dIdle) / float64(dTotal)), nil
}
// readCPUTicks parses the aggregate "cpu" line of /proc/stat:
// user nice system idle iowait irq softirq steal ...
func readCPUTicks() (idle, total uint64, err error) {
var buf [512]byte
f, err := os.Open("/proc/stat")
if err != nil {
return 0, 0, err
}
n, _ := f.Read(buf[:])
f.Close()
line := buf[:n]
if i := bytes.IndexByte(line, '\n'); i >= 0 {
line = line[:i]
}
fields := bytes.Fields(line)
if len(fields) < 5 || string(fields[0]) != "cpu" {
return 0, 0, errors.New("unexpected /proc/stat")
}
for i, f := range fields[1:] {
if i >= 8 { // guest time is already counted in user/nice
break
}
v, _ := strconv.ParseUint(string(f), 10, 64)
total += v
if i == 3 || i == 4 { // idle, iowait
idle += v
}
}
return idle, total, nil
}
// memUsage reads /proc/meminfo; used is everything not available to new
// processes (so page cache does not count as used).
func memUsage() (total, used uint64, err error) {
data, err := os.ReadFile("/proc/meminfo")
if err != nil {
return 0, 0, err
}
var avail, free uint64
var haveAvail bool
for len(data) > 0 {
line := data
if i := bytes.IndexByte(data, '\n'); i >= 0 {
line, data = data[:i], data[i+1:]
} else {
data = nil
}
switch {
case bytes.HasPrefix(line, []byte("MemTotal:")):
total = meminfoKB(line)
case bytes.HasPrefix(line, []byte("MemFree:")):
free = meminfoKB(line)
case bytes.HasPrefix(line, []byte("MemAvailable:")):
avail, haveAvail = meminfoKB(line), true
}
if haveAvail && total > 0 {
break
}
}
if total == 0 {
return 0, 0, errors.New("unexpected /proc/meminfo")
}
if !haveAvail {
avail = free
}
if avail > total {
avail = total
}
return total, total - avail, nil
}
// meminfoKB returns the value of a "Key: 123 kB" line in bytes.
func meminfoKB(line []byte) uint64 {
f := bytes.Fields(line)
if len(f) < 2 {
return 0
}
v, _ := strconv.ParseUint(string(f[1]), 10, 64)
return v * 1024
}
+33 -4
View File
@@ -17,9 +17,26 @@ func (a *Agent) runTask(ctx context.Context, taskID string) {
return
}
var res proto.TaskResult
if env.Type == proto.TaskSysUpdate {
switch env.Type {
case proto.TaskSysUpdate:
res = a.doUpdate()
} else {
case proto.TaskSysSpeed:
res = a.doSpeed(ctx, env)
case proto.TaskFSPut:
res = a.doFSPut(ctx, env)
case proto.TaskFSSend:
res = a.doFSSend(ctx, env)
case proto.TaskSSHFSSend:
res = a.doSSHFSSend(ctx, env)
case proto.TaskSSHFSPut:
res = a.doSSHFSPut(ctx, env)
case proto.TaskSSHFSOpen, proto.TaskSSHFSClose, proto.TaskSSHFSOp:
res = resultOf(a.sshfsTask(ctx, env))
case proto.TaskFSShipOut:
res = a.doShipOut(ctx, env)
case proto.TaskFSShipIn:
res = a.doShipIn(ctx, env)
default:
res = a.execTask(env)
}
a.postResult(ctx, taskID, res)
@@ -27,7 +44,7 @@ func (a *Agent) runTask(ctx context.Context, taskID string) {
func (a *Agent) fetchTask(ctx context.Context, taskID string) (proto.TaskEnvelope, error) {
var env proto.TaskEnvelope
req, _ := http.NewRequestWithContext(ctx, http.MethodGet, a.cfg.Host+"/api/agent/task/"+taskID, nil)
req, _ := http.NewRequestWithContext(ctx, http.MethodGet, a.host()+"/api/agent/task/"+taskID, nil)
a.authHeaders(req)
resp, err := a.http.Do(req)
if err != nil {
@@ -43,7 +60,7 @@ func (a *Agent) fetchTask(ctx context.Context, taskID string) (proto.TaskEnvelop
func (a *Agent) postResult(ctx context.Context, taskID string, res proto.TaskResult) {
body, _ := json.Marshal(res)
req, _ := http.NewRequestWithContext(ctx, http.MethodPost, a.cfg.Host+"/api/agent/task/"+taskID+"/result", bytes.NewReader(body))
req, _ := http.NewRequestWithContext(ctx, http.MethodPost, a.host()+"/api/agent/task/"+taskID+"/result", bytes.NewReader(body))
a.authHeaders(req)
req.Header.Set("Content-Type", "application/json")
resp, err := a.http.Do(req)
@@ -96,3 +113,15 @@ func itoa64(n int64) string {
}
return string(b[i:])
}
// resultOf makes a task's result of what it returned.
func resultOf(data any, err error) proto.TaskResult {
if err != nil {
return proto.TaskResult{Error: err.Error()}
}
raw, err := json.Marshal(data)
if err != nil {
return proto.TaskResult{Error: err.Error()}
}
return proto.TaskResult{OK: true, Data: raw}
}
+62
View File
@@ -0,0 +1,62 @@
package agent
import (
"errors"
"os"
"os/exec"
"runtime"
"strconv"
"strings"
"sync"
"golang.org/x/sys/unix"
)
// termsBySID are the running terminals by their session, so the panel can
// ask where one of them is.
var termsBySID sync.Map // sid -> *PTY
// Cwd is the folder the terminal is in now: that of the program in front
// (the shell, or what it runs), else of the shell itself.
func (p *PTY) Cwd() (string, error) {
pid := 0
if g, err := unix.IoctlGetInt(int(p.f.Fd()), unix.TIOCGPGRP); err == nil && g > 0 {
pid = g
}
if pid > 0 {
if d, err := procCwd(pid); err == nil {
return d, nil
}
}
return procCwd(p.cmd.Process.Pid)
}
// procCwd is a process's working folder: /proc on Linux, lsof elsewhere.
func procCwd(pid int) (string, error) {
if runtime.GOOS == "linux" {
return os.Readlink("/proc/" + strconv.Itoa(pid) + "/cwd")
}
out, err := exec.Command("lsof", "-a", "-p", strconv.Itoa(pid), "-d", "cwd", "-Fn").Output()
if err != nil {
return "", err
}
for _, l := range strings.Split(string(out), "\n") {
if strings.HasPrefix(l, "n/") {
return l[1:], nil
}
}
return "", errors.New("папка терминала не найдена")
}
// termCwd answers the panel: where the terminal of session sid is now.
func termCwd(sid string) (any, error) {
v, ok := termsBySID.Load(sid)
if !ok {
return nil, errors.New("терминал не найден")
}
d, err := v.(*PTY).Cwd()
if err != nil {
return nil, err
}
return map[string]string{"path": d}, nil
}
+87 -10
View File
@@ -4,6 +4,11 @@ import (
"fmt"
"os"
"os/exec"
"path/filepath"
"strings"
"sync"
"syscall"
"time"
"github.com/creack/pty"
)
@@ -12,16 +17,52 @@ import (
type PTY struct {
cmd *exec.Cmd
f *os.File
exited chan struct{} // closed once the shell has exited
once sync.Once
}
// hangupGrace is how long a shell gets after the hangup to save its history
// and exit before it is killed.
const hangupGrace = 3 * time.Second
// openPTYs are the terminals running now, to close on shutdown.
var openPTYs sync.Map // *PTY -> struct{}
// CloseTerminals hangs up every open terminal and waits for them, so the
// shells save their history before the agent exits.
func CloseTerminals() {
var wg sync.WaitGroup
openPTYs.Range(func(k, _ any) bool {
wg.Add(1)
go func() { defer wg.Done(); k.(*PTY).Close() }()
return true
})
wg.Wait()
}
// StartPTY launches a login shell attached to a new PTY. With user empty (or
// equal to the agent's own user) it is the agent's shell — root by default,
// equal to the agent's own user) it is the agent's shell - root by default,
// since the agent runs as a root systemd service. Any other user is entered via
// `su -l`, which only a root agent can do without a password.
func StartPTY(user string) (*PTY, error) {
// `su -l`, which only a root agent can do without a password. dir, when it
// is a folder, is where the shell starts instead of home.
func StartPTY(user, dir string) (*PTY, error) {
if dir != "" && (!filepath.IsAbs(dir) || strings.ContainsAny(dir, "\x00\r\n") || !isDir(dir)) {
dir = ""
}
var cmd *exec.Cmd
env := append(os.Environ(), "TERM=xterm-256color")
if user == "" || user == currentUser() {
cmd = exec.Command(defaultShell(), "-l")
// A login shell does not change directory by itself: it would start
// where the service runs, "/". Start it at home, and say where home
// is, as a root service often has no $HOME.
if home := homeDir(""); isDir(home) {
cmd.Dir = home
env = append(env, "HOME="+home)
}
if dir != "" {
cmd.Dir = dir
}
} else {
if !validUserName(user) {
return nil, fmt.Errorf("недопустимое имя пользователя %q", user)
@@ -37,13 +78,31 @@ func StartPTY(user string) (*PTY, error) {
return nil, fmt.Errorf("su не найден: %w", err)
}
cmd = exec.Command(su, "-l", user)
// su -l always goes home; to start elsewhere, its shell changes
// directory first and becomes the login shell (sh and csh alike).
if dir != "" {
cmd = exec.Command(su, "-l", user, "-c", "cd "+shQuote(dir)+"; exec \"$SHELL\" -l")
}
cmd.Env = append(os.Environ(), "TERM=xterm-256color")
// su -l goes to the user's home; starting there too keeps the shell
// out of "/" should it not.
if home := homeDir(user); isDir(home) {
cmd.Dir = home
}
}
cmd.Env = env
f, err := pty.Start(cmd)
if err != nil {
return nil, err
}
return &PTY{cmd: cmd, f: f}, nil
p := &PTY{cmd: cmd, f: f, exited: make(chan struct{})}
go func() { _ = cmd.Wait(); close(p.exited) }()
openPTYs.Store(p, struct{}{})
return p, nil
}
func isDir(p string) bool {
info, err := os.Stat(p)
return err == nil && info.IsDir()
}
// Read reads terminal output.
@@ -60,12 +119,30 @@ func (p *PTY) Resize(cols, rows uint16) {
_ = pty.Setsize(p.f, &pty.Winsize{Rows: rows, Cols: cols})
}
// Close terminates the shell and releases the PTY.
// Close hangs the terminal up as a closed window would: the shell gets
// SIGHUP, saves its history and exits. One that does not within hangupGrace
// is killed. Safe to call more than once.
func (p *PTY) Close() error {
err := p.f.Close()
if p.cmd.Process != nil {
_ = p.cmd.Process.Kill()
var err error
p.once.Do(func() {
// Closing the master hangs up the session; the signal to the
// shell's process group makes sure it hears, through su as well.
err = p.f.Close()
pid := p.cmd.Process.Pid
_ = syscall.Kill(-pid, syscall.SIGHUP)
select {
case <-p.exited:
case <-time.After(hangupGrace):
_ = syscall.Kill(-pid, syscall.SIGKILL)
<-p.exited
}
_ = p.cmd.Wait()
openPTYs.Delete(p)
})
return err
}
// shQuote puts s in single quotes for sh and csh alike, closing and
// reopening them around each quote inside.
func shQuote(s string) string {
return "'" + strings.ReplaceAll(s, "'", `'\''`) + "'"
}
+115
View File
@@ -0,0 +1,115 @@
package agent
import (
"io"
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/x-grk/longpolling-remote-access/internal/proto"
)
// A terminal opened with a folder starts its shell there; a folder that is
// not there leaves it at home.
func TestStartPTYDir(t *testing.T) {
dir, err := filepath.EvalSymlinks(t.TempDir())
if err != nil {
t.Fatal(err)
}
sub := filepath.Join(dir, "sub dir")
if err := os.Mkdir(sub, 0o755); err != nil {
t.Fatal(err)
}
if got := ptyPwd(t, sub); got != sub {
t.Fatalf("pwd = %q, want %q", got, sub)
}
home, _ := filepath.EvalSymlinks(homeDir(""))
if got := ptyPwd(t, filepath.Join(dir, "missing")); got != home {
t.Fatalf("pwd for a missing folder = %q, want home %q", got, home)
}
}
// ptyPwd opens a terminal at dir and returns where its shell is.
func ptyPwd(t *testing.T, dir string) string {
t.Helper()
t.Setenv("SHELL", "/bin/sh")
p, err := StartPTY("", dir)
if err != nil {
t.Fatal(err)
}
defer p.Close()
if _, err := p.Write([]byte("echo \"@@$(pwd -P)@@\"\n")); err != nil {
t.Fatal(err)
}
var out strings.Builder
buf := make([]byte, 4096)
deadline := time.Now().Add(5 * time.Second)
for time.Now().Before(deadline) {
n, err := p.Read(buf)
out.Write(buf[:n])
s := out.String()
// The echoed command line has "@@$(pwd" in it; the answer does not.
for _, part := range strings.Split(s, "@@") {
if strings.HasPrefix(part, "/") && strings.Contains(s, part+"@@") {
return part
}
}
if err != nil {
break
}
}
t.Fatalf("no pwd in %q", out.String())
return ""
}
// A terminal tells the folder its shell is in now, after a cd.
func TestPTYCwd(t *testing.T) {
dir, err := filepath.EvalSymlinks(t.TempDir())
if err != nil {
t.Fatal(err)
}
sub := filepath.Join(dir, "next one")
if err := os.Mkdir(sub, 0o755); err != nil {
t.Fatal(err)
}
t.Setenv("SHELL", "/bin/sh")
p, err := StartPTY("", dir)
if err != nil {
t.Fatal(err)
}
defer p.Close()
go func() { _, _ = io.Copy(io.Discard, p) }()
if _, err := p.Write([]byte("cd 'next one'\n")); err != nil {
t.Fatal(err)
}
var got string
for i := 0; i < 50; i++ {
if got, err = p.Cwd(); err == nil && got == sub {
return
}
time.Sleep(100 * time.Millisecond)
}
t.Fatalf("Cwd = %q, %v; want %q", got, err, sub)
}
// The panel asks for a terminal's folder by its session.
func TestTermCwdTask(t *testing.T) {
dir, _ := filepath.EvalSymlinks(t.TempDir())
t.Setenv("SHELL", "/bin/sh")
p, err := StartPTY("", dir)
if err != nil {
t.Fatal(err)
}
defer p.Close()
termsBySID.Store("s1", p)
defer termsBySID.Delete("s1")
res, err := dispatchTask(proto.TaskEnvelope{Type: proto.TaskTermCwd, Args: []byte(`{"session":"s1"}`)})
if err != nil || res.(map[string]string)["path"] != dir {
t.Fatalf("term.cwd = %v, %v; want %q", res, err, dir)
}
if _, err := dispatchTask(proto.TaskEnvelope{Type: proto.TaskTermCwd, Args: []byte(`{"session":"nope"}`)}); err == nil {
t.Fatal("an unknown session has a folder")
}
}
+25 -19
View File
@@ -15,33 +15,33 @@ import (
// --- direct TCP mux tunnel ---
// runTunnel keeps a persistent connection to the host's 8446 tunnel, dialing
// client-side ports on demand for "tcp" forward rules.
func (a *Agent) runTunnel(ctx context.Context) {
// runTunnel keeps a persistent connection to the host's tunnel at addr, dialing
// client-side ports on demand for "tcp" forward rules, until ctx is cancelled.
func (a *Agent) runTunnel(ctx context.Context, addr string) {
backoff := time.Second
for ctx.Err() == nil {
if err := a.tunnelOnce(ctx); err != nil {
if err := a.tunnelOnce(ctx, addr); err != nil {
select {
case <-time.After(backoff):
case <-ctx.Done():
return
}
if backoff < 15*time.Second {
backoff *= 2
}
backoff = min(backoff*2, retryCap)
continue
}
backoff = time.Second
}
}
func (a *Agent) tunnelOnce(ctx context.Context) error {
func (a *Agent) tunnelOnce(ctx context.Context, addr string) error {
d := net.Dialer{Timeout: 10 * time.Second}
conn, err := d.DialContext(ctx, "tcp", a.cfg.TunnelAddr)
conn, err := d.DialContext(ctx, "tcp", addr)
if err != nil {
return err
}
defer conn.Close()
// Unblock the read loop when the tunnel is stopped or moved.
defer context.AfterFunc(ctx, func() { conn.Close() })()
if tc, ok := conn.(*net.TCPConn); ok {
_ = tc.SetNoDelay(true)
}
@@ -65,7 +65,7 @@ func (a *Agent) tunnelOnce(ctx context.Context) error {
return err
}
_ = conn.SetDeadline(time.Time{})
log.Printf("tunnel: connected to %s", a.cfg.TunnelAddr)
log.Printf("tunnel: connected to %s", addr)
tc := &tunClient{conn: conn, streams: map[uint32]*agentStream{}}
return tc.readLoop()
@@ -94,8 +94,14 @@ func (t *tunClient) write(f tunnel.Frame) error {
return tunnel.WriteFrame(t.conn, f)
}
// tunnelSilence is how long the tunnel may say nothing before it is taken
// for dead: the server pings every 20 s, so this is three missed pings - a
// connection a NAT forgot while the machine slept, say.
const tunnelSilence = 65 * time.Second
func (t *tunClient) readLoop() error {
for {
_ = t.conn.SetReadDeadline(time.Now().Add(tunnelSilence))
f, err := tunnel.ReadFrame(t.conn)
if err != nil {
t.shutdown()
@@ -194,28 +200,26 @@ func (t *tunClient) shutdown() {
// --- direct UDP forwarding ---
// runUDP maintains the agent's UDP endpoint on the host's 8446 channel and
// runUDP maintains the agent's UDP endpoint on the host's tunnel at addr and
// relays datagrams to/from client-side UDP ports for "udp" forward rules.
func (a *Agent) runUDP(ctx context.Context) {
func (a *Agent) runUDP(ctx context.Context, addr string) {
backoff := time.Second
for ctx.Err() == nil {
if err := a.udpOnce(ctx); err != nil {
if err := a.udpOnce(ctx, addr); err != nil {
select {
case <-time.After(backoff):
case <-ctx.Done():
return
}
if backoff < 15*time.Second {
backoff *= 2
}
backoff = min(backoff*2, retryCap)
continue
}
backoff = time.Second
}
}
func (a *Agent) udpOnce(ctx context.Context) error {
raddr, err := net.ResolveUDPAddr("udp", a.cfg.TunnelAddr)
func (a *Agent) udpOnce(ctx context.Context, addr string) error {
raddr, err := net.ResolveUDPAddr("udp", addr)
if err != nil {
return err
}
@@ -224,6 +228,8 @@ func (a *Agent) udpOnce(ctx context.Context) error {
return err
}
defer conn.Close()
// Unblock the read loop when the tunnel is stopped or moved.
defer context.AfterFunc(ctx, func() { conn.Close() })()
key := tunnel.AuthKey(a.state.Token)
register := func() {
@@ -268,7 +274,7 @@ func (a *Agent) udpOnce(ctx context.Context) error {
}
type udpClient struct {
server *net.UDPConn // connection back to host:8446
server *net.UDPConn // connection back to the host tunnel
mu sync.Mutex
dials map[uint32]*net.UDPConn // sid -> client-side socket
}
+117 -24
View File
@@ -3,31 +3,40 @@ package agent
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"log"
"net/http"
"os"
"os/exec"
"path/filepath"
"runtime"
"strings"
"syscall"
"time"
"github.com/x-grk/longpolling-remote-access/internal/proto"
)
// doUpdate downloads the latest agent binary for this platform, atomically
// replaces the running executable, and schedules a service restart. It never
// uninstalls — only updates.
// doUpdate downloads the latest agent binary for this platform, replaces the
// running executable and restarts the service. It runs as root when it can:
// the agent's own user if that is root, otherwise sudo when it needs no
// password. It never uninstalls - only updates.
func (a *Agent) doUpdate() proto.TaskResult {
exe, err := os.Executable()
if err != nil {
return proto.TaskResult{Error: "не удалось определить путь бинарника: " + err.Error()}
}
exe, _ = filepath.EvalSymlinks(exe)
sudo := ""
if os.Geteuid() != 0 {
sudo = sudoPath()
}
url := fmt.Sprintf("%s/download/agent-%s-%s", a.cfg.Host, runtime.GOOS, runtime.GOARCH)
url := fmt.Sprintf("%s/download/agent-%s-%s", a.host(), runtime.GOOS, runtime.GOARCH)
req, _ := http.NewRequestWithContext(context.Background(), http.MethodGet, url, nil)
resp, err := a.http.Do(req)
resp, err := a.stream.Do(req)
if err != nil {
return proto.TaskResult{Error: "скачивание: " + err.Error()}
}
@@ -36,41 +45,125 @@ func (a *Agent) doUpdate() proto.TaskResult {
return proto.TaskResult{Error: "скачивание: сервер вернул " + resp.Status}
}
tmp := exe + ".new"
f, err := os.OpenFile(tmp, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0o755)
if err != nil {
return proto.TaskResult{Error: "создание файла: " + err.Error()}
// The new binary lands next to the old one, or, when root's hand is
// needed to put it there, in a temporary folder first.
dir := filepath.Dir(exe)
if sudo != "" {
dir = ""
}
if _, err := io.Copy(f, resp.Body); err != nil {
f.Close()
os.Remove(tmp)
f, err := os.CreateTemp(dir, ".lpagent-new-*")
if err != nil {
return proto.TaskResult{Error: "создание файла: " + err.Error() + userNote()}
}
tmp := f.Name()
defer os.Remove(tmp)
n, err := io.Copy(f, io.LimitReader(resp.Body, 256<<20))
if cerr := f.Close(); err == nil {
err = cerr
}
if err != nil {
return proto.TaskResult{Error: "запись: " + err.Error()}
}
f.Close()
if err := checkBinary(tmp, n); err != nil {
return proto.TaskResult{Error: err.Error()}
}
if err := os.Chmod(tmp, 0o755); err != nil {
return proto.TaskResult{Error: err.Error()}
}
// Atomic replace: renaming over a running executable is allowed on Linux;
// the live process keeps the old inode until it restarts.
if err := os.Rename(tmp, exe); err != nil {
os.Remove(tmp)
return proto.TaskResult{Error: "замена бинарника: " + err.Error()}
how := "от " + currentUser()
if sudo != "" {
// install copies it in place under a new inode, as a rename would.
if out, err := exec.Command(sudo, "-n", "install", "-m", "0755", tmp, exe).CombinedOutput(); err != nil {
return proto.TaskResult{Error: "замена бинарника через sudo: " + strings.TrimSpace(string(out)+" "+err.Error())}
}
how = "через sudo"
} else if err := os.Rename(tmp, exe); err != nil {
// Renaming over a running executable is allowed; the live process
// keeps the old inode until it restarts.
return proto.TaskResult{Error: "замена бинарника: " + err.Error() + userNote()}
}
// Restart shortly, after the result has been reported.
go func() {
time.Sleep(1500 * time.Millisecond)
restartSelf()
restartSelf(exe, sudo)
}()
data, _ := json.Marshal(map[string]string{"status": "обновлено, перезапуск"})
data, _ := json.Marshal(map[string]string{"status": "обновлено " + how + ", перезапуск"})
return proto.TaskResult{OK: true, Data: data}
}
// restartSelf restarts the agent so the new binary takes effect.
func restartSelf() {
// sudoPath is sudo when it lets this user run commands without a password.
func sudoPath() string {
p, err := exec.LookPath("sudo")
if err != nil {
return ""
}
if exec.Command(p, "-n", "true").Run() != nil {
return ""
}
return p
}
func userNote() string {
if os.Geteuid() == 0 {
return ""
}
return " (агент запущен от " + currentUser() + ", без root и sudo без пароля - обновите установщиком)"
}
// checkBinary makes sure what came is a program for this system, not an
// error page: a binary of a sensible size with the right header.
func checkBinary(path string, size int64) error {
if size < 1<<20 {
return fmt.Errorf("скачанный файл слишком мал (%d байт) - не похоже на агента", size)
}
f, err := os.Open(path)
if err != nil {
return err
}
defer f.Close()
var magic [4]byte
if _, err := io.ReadFull(f, magic[:]); err != nil {
return err
}
ok := false
switch runtime.GOOS {
case "linux":
ok = string(magic[:]) == "\x7fELF"
case "darwin":
ok = magic == [4]byte{0xcf, 0xfa, 0xed, 0xfe} || magic == [4]byte{0xca, 0xfe, 0xba, 0xbe}
default:
ok = true
}
if !ok {
return errors.New("скачанный файл - не программа для " + runtime.GOOS)
}
return nil
}
// restartSelf starts the new binary. Under systemd the service is restarted
// (queued, so it goes on after this process is stopped); otherwise the
// process replaces itself with the new binary.
func restartSelf(exe, sudo string) {
if os.Getenv("INVOCATION_ID") != "" {
if path, err := exec.LookPath("systemctl"); err == nil {
_ = exec.Command(path, "restart", "lpagent").Start()
args := []string{"--no-block", "restart", "lpagent"}
cmd := exec.Command(path, args...)
if sudo != "" {
cmd = exec.Command(sudo, append([]string{"-n", path}, args...)...)
}
if cmd.Run() == nil {
return
}
// No systemd (e.g. launchd/manual run): exit so the supervisor restarts us.
}
// systemd brings a service that stopped back up (Restart=always).
os.Exit(0)
}
CloseTerminals()
if err := syscall.Exec(exe, os.Args, os.Environ()); err != nil {
log.Printf("update: restart: %v", err)
os.Exit(0)
}
}
+8
View File
@@ -33,10 +33,18 @@ func dispatchTask(env proto.TaskEnvelope) (any, error) {
switch env.Type {
case proto.TaskSysUsers:
return sysUsers(), nil
case proto.TaskSysStat:
return sysStat()
case proto.TaskSysUserAdd:
var a proto.SysUserAddArgs
_ = json.Unmarshal(env.Args, &a)
return nil, addUser(a)
case proto.TaskDBTables, proto.TaskDBRows, proto.TaskDBSet, proto.TaskDBAdd, proto.TaskDBDel, proto.TaskDBSQL:
return dbTask(env)
case proto.TaskTermCwd:
var a proto.TermCwdArgs
_ = json.Unmarshal(env.Args, &a)
return termCwd(a.Session)
}
return dispatchFS(env)
}
+179
View File
@@ -0,0 +1,179 @@
package agent
import (
"context"
"errors"
"fmt"
"log"
"net/http"
"strings"
"sync"
"time"
"github.com/coder/websocket"
"github.com/coder/websocket/wsjson"
"github.com/x-grk/longpolling-remote-access/internal/proto"
)
// The WebSocket to the server: control messages arrive on it as soon as they
// are queued, and terminal and http-forward sessions send and take their
// bytes over it instead of a request each. When it drops, sessions carry on
// over plain requests.
const wsPing = 20 * time.Second
type wsLink struct {
c *websocket.Conn
ctx context.Context
mu sync.Mutex
subs map[string]chan wsIn
asked map[string]bool // sessions whose input was asked for
done chan struct{}
stop context.CancelFunc
}
// close drops the link: the agent then connects again, to the address in
// use by then.
func (l *wsLink) close() { l.stop() }
// wsIn is what the server sent for a session: its input, or that it is over.
type wsIn struct {
frames []proto.InFrame
gone bool
}
func (l *wsLink) send(m proto.WSMsg) error {
ctx, cancel := context.WithTimeout(l.ctx, 30*time.Second)
defer cancel()
return wsjson.Write(ctx, l.c, m)
}
// subscribe asks the server for a session's input on this link, once, and
// returns where it arrives; nil when the link cannot take it.
func (l *wsLink) subscribe(sid string) chan wsIn {
l.mu.Lock()
ch := l.subs[sid]
if ch == nil {
ch = make(chan wsIn, 64)
l.subs[sid] = ch
}
fresh := !l.subscribed(sid)
l.mu.Unlock()
if fresh && l.send(proto.WSMsg{T: "sub", SID: sid}) != nil {
return nil
}
return ch
}
// subscribed marks a session as asked for; l.mu is held.
func (l *wsLink) subscribed(sid string) bool {
if l.asked == nil {
l.asked = map[string]bool{}
}
if l.asked[sid] {
return true
}
l.asked[sid] = true
return false
}
func (l *wsLink) unsubscribe(sid string) {
l.mu.Lock()
delete(l.subs, sid)
delete(l.asked, sid)
l.mu.Unlock()
}
func (l *wsLink) deliver(sid string, in wsIn) {
l.mu.Lock()
ch := l.subs[sid]
l.mu.Unlock()
if ch == nil {
return
}
select {
case ch <- in:
case <-l.done:
}
}
// runWS holds the WebSocket until it ends, dispatching what comes on it.
func (a *Agent) runWS(ctx context.Context) error {
u := a.host() + "/api/agent/ws"
u = "ws" + strings.TrimPrefix(u, "http")
hdr := http.Header{}
hdr.Set("X-Agent-Id", a.state.AgentID)
hdr.Set("Authorization", "Bearer "+a.state.Token)
hdr.Set("X-Agent-Version", Version)
hdr.Set("X-Agent-User", currentUser())
dctx, dcancel := context.WithTimeout(ctx, 15*time.Second)
c, resp, err := websocket.Dial(dctx, u, &websocket.DialOptions{
HTTPClient: a.stream, HTTPHeader: hdr, CompressionMode: websocket.CompressionDisabled,
})
dcancel()
if err != nil {
if resp != nil {
a.applyTransport(resp.Header.Get(proto.TransportHeader))
return fmt.Errorf("server answered %s", resp.Status)
}
return err
}
c.SetReadLimit(8 << 20)
lctx, lcancel := context.WithCancel(ctx)
l := &wsLink{c: c, ctx: lctx, subs: map[string]chan wsIn{}, done: make(chan struct{}), stop: lcancel}
a.link.Store(l)
defer func() {
a.link.CompareAndSwap(l, nil)
close(l.done)
lcancel()
c.CloseNow()
}()
log.Printf("connected by websocket")
a.reached()
go func() {
t := time.NewTicker(wsPing)
defer t.Stop()
for {
select {
case <-lctx.Done():
return
case <-t.C:
pctx, pcancel := context.WithTimeout(lctx, 15*time.Second)
err := c.Ping(pctx)
pcancel()
if err != nil {
lcancel()
return
}
}
}
}()
for {
var m proto.WSMsg
if err := wsjson.Read(lctx, c, &m); err != nil {
if errors.Is(err, context.Canceled) && ctx.Err() == nil {
return errors.New("no answer to ping")
}
return err
}
switch m.T {
case "ctrl":
if m.Ctrl != nil {
// Sessions live on the agent's context: they outlast the link.
a.dispatch(ctx, *m.Ctrl)
}
case "cfg":
a.applyTunnelAddr(ctx, m.Tunnel)
a.learnAddrs(m.Main, m.Fallback)
a.applyTransport(m.Transport)
if a.mode() == proto.TransportLP {
return errors.New("set to long polling")
}
case "in":
l.deliver(m.SID, wsIn{frames: m.Frames})
case "gone":
l.deliver(m.SID, wsIn{gone: true})
}
}
}
+251 -5
View File
@@ -1,15 +1,17 @@
// Package proto describes the wire messages shared by the server and the agent.
//
// All agent<->server traffic is plain JSON over HTTP(S) long-polling so it
// survives NAT and restrictive egress firewalls: the agent only ever makes
// outbound requests.
// All agent<->server traffic is plain JSON, on a WebSocket the agent opens or,
// where that does not get through, over HTTP(S) long polls, so it survives
// NAT and restrictive egress firewalls: the agent only ever connects out.
package proto
import "strings"
import "encoding/json"
// Control poll message types (server -> agent, returned from /api/agent/poll).
const (
// CtrlIdle is returned when the long-poll times out with nothing to do.
// CtrlIdle is returned when the control poll times out with nothing to do.
CtrlIdle = "idle"
// CtrlOpenTerminal tells the agent to spawn a PTY for the session id.
CtrlOpenTerminal = "open_terminal"
@@ -18,8 +20,26 @@ const (
CtrlOpenForward = "open_forward"
// CtrlTask tells the agent to fetch and run the one-shot task TaskID.
CtrlTask = "task"
// CtrlOpenSSH tells the agent to open an SSH connection to SSH and bridge
// its shell to the session id, as a terminal.
CtrlOpenSSH = "open_ssh"
)
// SSHTarget is where an SSH terminal connects and how it signs in: a
// password, a private key (PEM or OpenSSH, with its passphrase if it has
// one), or, with neither, the agent user's own keys in ~/.ssh.
type SSHTarget struct {
Host string `json:"host"`
Port int `json:"port,omitempty"`
User string `json:"user"`
Password string `json:"password,omitempty"`
Key string `json:"key,omitempty"`
Passphrase string `json:"passphrase,omitempty"`
// Saved: a host the user saved on the server; it fills the rest there and
// never reaches an agent.
Saved int64 `json:"saved,omitempty"`
}
// EnrollRequest is sent once by a freshly installed agent.
type EnrollRequest struct {
Name string `json:"name"`
@@ -36,13 +56,53 @@ type EnrollResponse struct {
Token string `json:"token"`
}
// ControlMessage is the body returned by the control long-poll.
// ControlMessage is a command for the agent: on the WebSocket, or the body of
// the control long poll.
type ControlMessage struct {
Type string `json:"type"`
SessionID string `json:"session_id,omitempty"`
Port int `json:"port,omitempty"` // CtrlOpenForward: client-side port to dial
TaskID string `json:"task_id,omitempty"` // CtrlTask
User string `json:"user,omitempty"` // CtrlOpenTerminal: OS user for the shell ("" = the agent's own)
Dir string `json:"dir,omitempty"` // CtrlOpenTerminal: the folder to start in ("" or missing = home)
SSH *SSHTarget `json:"ssh,omitempty"` // CtrlOpenSSH
}
// Transports an agent can reach the server by.
const (
TransportAuto = "auto" // WebSocket, long polling while it cannot connect
TransportWS = "ws" // WebSocket only
TransportLP = "lp" // long polling only
)
// TransportHeader carries the agent's transport setting on poll and config
// answers.
const TransportHeader = "X-Transport"
// FallbackHeader carries the server's direct address (one not behind a
// proxy like Cloudflare) on poll and config answers, for the agent to turn
// to when the main one is out of reach.
const FallbackHeader = "X-Fallback-Host"
// MainHeader carries the server's main address beside FallbackHeader, for an
// agent set to the direct one to fall back to the main one.
const MainHeader = "X-Main-Host"
// WSMsg is one message on an agent's WebSocket, either way. Server to agent:
// "ctrl" (Ctrl, as a control poll would answer), "cfg" (Tunnel and
// Transport, also a keep-alive), "in" (Frames for session SID), "gone"
// (session SID is over). Agent to server: "sub" (send the input of session
// SID here), "out" (Out of session SID).
type WSMsg struct {
T string `json:"t"`
Ctrl *ControlMessage `json:"ctrl,omitempty"`
SID string `json:"sid,omitempty"`
Frames []InFrame `json:"frames,omitempty"`
Out *OutMessage `json:"out,omitempty"`
Tunnel string `json:"tunnel,omitempty"`
Transport string `json:"transport,omitempty"`
Fallback string `json:"fallback,omitempty"`
Main string `json:"main,omitempty"`
}
// Terminal / stream frame types on the session "in" channel (server -> agent).
@@ -70,8 +130,20 @@ type InBatch struct {
type OutMessage struct {
Data string `json:"data,omitempty"` // base64 bytes
Exit bool `json:"exit,omitempty"`
// Status tells the panel how an SSH terminal's connection goes: one of
// the SSHStatus values, with Note to show.
Status string `json:"status,omitempty"`
Note string `json:"note,omitempty"`
}
// The steps of an SSH terminal's connection, as OutMessage.Status.
const (
SSHDial = "dial" // connecting to the host
SSHAuth = "auth" // the host answered, signing in
SSHOK = "ok" // signed in, the shell runs
SSHErr = "err" // it failed; Note says why
)
// --- one-shot tasks (file manager, native in the agent) ---
const (
@@ -85,12 +157,62 @@ const (
TaskFSZip = "fs.zip" // zip selected paths (native archive/zip)
TaskFSMove = "fs.move" // move/rename
TaskFSDownload = "fs.download" // read a whole file (larger cap) for browser download
TaskFSStat = "fs.stat" // a file's properties: type, size, mode, owner
TaskFSChmod = "fs.chmod" // change mode and owner, of a folder's contents too
TaskFSCopy = "fs.copy" // copy or move, with a rule for what is already there
TaskFSPut = "fs.put" // write a file the browser sends in pieces
TaskFSSend = "fs.send" // send a file to the browser in pieces
TaskSysUpdate = "sys.update" // download the latest agent binary and restart
TaskSysUsers = "sys.users" // report the agent's user and the login-capable users
TaskSysUserAdd = "sys.useradd" // create a new OS user
TaskSysStat = "sys.stat" // CPU load, RAM and disk usage right now
TaskSysSpeed = "sys.speed" // measure the link to the server (ping, down, up)
TaskDBTables = "db.tables" // an SQLite database's tables and views
TaskDBRows = "db.rows" // a page of a table's rows, with each row's key
TaskDBSet = "db.set" // set one value of one row
TaskDBAdd = "db.add" // add a row
TaskDBDel = "db.del" // delete rows by their keys
TaskDBSQL = "db.sql" // run a statement typed in the panel
TaskTermCwd = "term.cwd" // the folder a terminal is in now (TermCwdArgs); result {path}
TaskSSHFSOpen = "sshfs.open" // connect to an SSH host for a file pane; result {id, home}
TaskSSHFSClose = "sshfs.close" // drop that connection
TaskSSHFSOp = "sshfs.op" // a file operation on the host: list, read, write, mkdir, delete, deltree, move, copy, stat, chmod
TaskSSHFSSend = "sshfs.send" // a file of the host to the browser in pieces (FSXferArgs with Conn)
TaskSSHFSPut = "sshfs.put" // a file from the browser to the host in pieces
TaskFSShipOut = "fs.ship.out" // pack a file or folder and send it to another machine
TaskFSShipIn = "fs.ship.in" // unpack what another machine sends
)
// FSReadCap bounds how many bytes TaskFSRead returns (5 MiB). The panel lets a
// file be edited only when it fits, so that a save never cuts it short.
const FSReadCap = 5 * 1024 * 1024
// FSFileCap bounds a file sent either way in pieces (TaskFSPut, TaskFSSend).
const FSFileCap = 512 << 20
// FSXferArgs are the arguments of TaskFSPut and TaskFSSend: the file, and the
// transfer on the server whose pieces carry it - GET them for a put, POST them
// for a send, Chunk bytes each. Size is a put's length; -1 for a send.
type FSXferArgs struct {
Path string `json:"path"`
Xfer string `json:"xfer"`
Size int64 `json:"size"`
Chunk int `json:"chunk"`
// Conn is the SSH file connection the file is on (sshfs.send, sshfs.put).
Conn string `json:"conn,omitempty"`
}
// SSHFSArgs are the arguments of the sshfs.* tasks: the connection (Conn), or
// where to connect (SSH, for open); for an operation its name (Op) and its
// arguments (Args) in the shapes of the fs.* tasks.
type SSHFSArgs struct {
Conn string `json:"conn,omitempty"`
SSH *SSHTarget `json:"ssh,omitempty"`
Op string `json:"op,omitempty"`
Args json.RawMessage `json:"args,omitempty"`
}
// TaskEnvelope is what the agent fetches for a task id.
type TaskEnvelope struct {
Type string `json:"type"`
@@ -125,6 +247,47 @@ type FSMoveArgs struct {
Dst string `json:"dst"`
}
// FSStatResult is returned for TaskFSStat. Mode holds the permission bits
// with setuid, setgid and sticky (07777). Users and Groups are the names on
// the machine, offered when choosing an owner.
type FSStatResult struct {
Path string `json:"path"`
IsDir bool `json:"is_dir"`
Link string `json:"link,omitempty"` // where a symlink points
Size int64 `json:"size"`
MTime int64 `json:"mtime"`
Mode uint32 `json:"mode"`
UID int `json:"uid"`
GID int `json:"gid"`
Owner string `json:"owner"`
Group string `json:"group"`
Users []string `json:"users"`
Groups []string `json:"groups"`
}
// FSChmodArgs are the arguments of TaskFSChmod. Mode nil, Owner or Group
// empty leave that as it is. Recursive applies to everything inside a folder
// as well, Only narrowing it to "files" or "dirs".
type FSChmodArgs struct {
Path string `json:"path"`
Mode *uint32 `json:"mode,omitempty"`
Owner string `json:"owner,omitempty"`
Group string `json:"group,omitempty"`
Recursive bool `json:"recursive,omitempty"`
Only string `json:"only,omitempty"`
}
// FSCopyArgs are the arguments of TaskFSCopy: Src goes to Dst (the full new
// path), copied or, with Move, moved. Conflict says what to do when Dst is
// already there: "error" (the default) stops, "replace" puts Src in its place,
// "skip" keeps it. Folders on both sides are merged under replace and skip.
type FSCopyArgs struct {
Src string `json:"src"`
Dst string `json:"dst"`
Move bool `json:"move,omitempty"`
Conflict string `json:"conflict,omitempty"`
}
// FSEntry is one directory entry.
type FSEntry struct {
Name string `json:"name"`
@@ -132,6 +295,8 @@ type FSEntry struct {
Size int64 `json:"size"`
Mode string `json:"mode"`
MTime int64 `json:"mtime"`
// Link: a symbolic link (IsDir then says where it leads: a folder).
Link bool `json:"link,omitempty"`
}
// FSListResult is returned for TaskFSList.
@@ -162,3 +327,84 @@ type SysUserAddArgs struct {
Password string `json:"password,omitempty"` // empty leaves the account without a password
Sudo bool `json:"sudo"` // add to the sudo/wheel group
}
// SysStatResult is returned for TaskSysStat. Sizes are in bytes.
type SysStatResult struct {
CPU float64 `json:"cpu"` // load across all cores, 0..100
NCPU int `json:"ncpu"`
MemTotal uint64 `json:"mem_total"`
MemUsed uint64 `json:"mem_used"`
DiskTotal uint64 `json:"disk_total"` // filesystem holding "/"
DiskFree uint64 `json:"disk_free"`
}
// SysSpeedArgs are the arguments of TaskSysSpeed.
type SysSpeedArgs struct {
// Relay makes the agent use the relayed streams, which the server pipes
// to and from the browser.
Relay bool `json:"relay,omitempty"`
// WindowMs is how long to measure each direction for; 0 leaves it to the
// agent (3 s, all an agent that predates this argument does).
WindowMs int `json:"window_ms,omitempty"`
}
// SysSpeedResult is returned for TaskSysSpeed: the agent's HTTP link to the
// server as the agent measured it. Down is server -> agent.
type SysSpeedResult struct {
PingMs float64 `json:"ping_ms"`
DownBps float64 `json:"down_bps"` // bits per second
UpBps float64 `json:"up_bps"`
Relay bool `json:"relay,omitempty"` // the streams went through the relay
// Partial marks a test the server cut short: the numbers are what it had
// counted on its own side by then.
Partial bool `json:"partial,omitempty"`
}
// FSShipArgs are the arguments of TaskFSShipOut and TaskFSShipIn: Src (on the
// sending machine) lands in Dir as Name (on the receiving one), following
// Conflict as FSCopyArgs does. The sender packs it as a tar stream and either
// posts it in pieces of Chunk bytes to Relay on the server, for the receiver
// to take, or with SSH set writes it to tar on that host itself, and with
// Move then deletes Src. Both report to Job how far they are.
type FSShipArgs struct {
Src string `json:"src,omitempty"`
Name string `json:"name"`
Dir string `json:"dir"`
Conflict string `json:"conflict,omitempty"`
Move bool `json:"move,omitempty"`
Relay string `json:"relay,omitempty"`
Chunk int `json:"chunk,omitempty"`
Job string `json:"job,omitempty"`
SSH *SSHTarget `json:"ssh,omitempty"`
// Conn: the files of this side are on the SSH host of this agent's file
// connection (an SSH file pane), not on the machine itself.
Conn string `json:"conn,omitempty"`
// More: other things of Src's folder sent in the same stream under their
// own names (several files dragged at once). Agents before 0.2.0 do not
// know it; the server sends to them one at a time.
More []string `json:"more,omitempty"`
}
// ShipNames is what lands at the other end: Name, then More by their names.
func (a FSShipArgs) ShipNames() []string {
n := []string{a.Name}
for _, p := range a.More {
p = strings.TrimRight(p, "/")
n = append(n, p[strings.LastIndex(p, "/")+1:])
}
return n
}
// ShipProgress is what a sending machine reports: bytes of files sent out of
// Total, and files done.
type ShipProgress struct {
Total int64 `json:"total"`
Done int64 `json:"done"`
Files int `json:"files"`
Note string `json:"note,omitempty"` // what is going on: connecting, signing in
}
// TermCwdArgs names the terminal session whose folder the panel asks for.
type TermCwdArgs struct {
Session string `json:"session"`
}
+10
View File
@@ -24,6 +24,16 @@ func (s *Server) authenticateAgent(r *http.Request) *Agent {
return a
}
// online reports whether the agent is connected right now: the hub knows from
// its control polls. Right after a start it cannot tell yet, so the last poll
// recorded in the database decides.
func (s *Server) online(a *Agent) bool {
if on, ok := s.hub.Online(a.ID); ok {
return on
}
return a.recentlySeen()
}
func bearerToken(r *http.Request) string {
h := r.Header.Get("Authorization")
if strings.HasPrefix(h, "Bearer ") {
File diff suppressed because it is too large. Load diff
+435
View File
@@ -0,0 +1,435 @@
package server
import (
"context"
"encoding/base64"
"encoding/json"
"fmt"
"io"
"net/http"
"net/http/httptest"
"strconv"
"strings"
"testing"
"time"
)
func collect(t *testing.T, cfg aiConfig) (string, error) {
t.Helper()
var b strings.Builder
err := aiStream(context.Background(), cfg, "sys", []aiMsg{{Role: "user", Content: "раз"}, {Role: "assistant", Content: "два"}, {Role: "user", Content: "три"}}, func(s string) { b.WriteString(s) })
return b.String(), err
}
// Gradio (the Qwen space behind the proxy): history in, the answer streamed
// as diffs of the chatbot, the key in X-Agent-Key.
func TestAIGradio(t *testing.T) {
var joined map[string]any
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get("X-Agent-Key") != "k" {
http.Error(w, "no key", 403)
return
}
switch r.URL.Path {
case "/config":
fmt.Fprint(w, `{"dependencies":[{"id":3,"api_name":"other"},{"id":0,"api_name":"chat_predict","targets":[[21,"click"]]}]}`)
case "/gradio_api/queue/join":
_ = json.NewDecoder(r.Body).Decode(&joined)
fmt.Fprint(w, `{"event_id":"e"}`)
case "/gradio_api/queue/data":
w.Header().Set("Content-Type", "text/event-stream")
for _, m := range []string{
`{"msg":"process_starts"}`,
`{"msg":"process_generating","output":{"data":[null,null,null,null,[{"role":"user","content":"три"}],{},{}]}}`,
`{"msg":"process_generating","output":{"data":[[],[],[],[],[["add",[3],{"role":"assistant","content":"При"}]],[],[]]}}`,
`{"msg":"process_generating","output":{"data":[[],[],[],[],[["append",[3,"content"],"вет"]],[],[]]}}`,
`{"msg":"process_completed","success":true,"output":{"data":[{},{},{},{},[{"role":"user","content":"три"},{"role":"assistant","content":"Привет!"}],{},{}]}}`,
} {
fmt.Fprintf(w, "data: %s\n\n", m)
}
default:
http.NotFound(w, r)
}
}))
defer srv.Close()
got, err := collect(t, aiConfig{Type: "gradio", URL: srv.URL, Key: "k"})
if err != nil || got != "Привет!" {
t.Fatalf("got %q, %v", got, err)
}
d := joined["data"].([]any)
if joined["fn_index"].(float64) != 0 || d[0] != "три" || d[5] != "sys" || len(d[4].([]any)) != 2 {
t.Fatalf("join: %v", joined)
}
}
func TestAIGradioError(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/config":
fmt.Fprint(w, `{"dependencies":[{"id":0,"api_name":"chat_predict"}]}`)
case "/gradio_api/queue/join":
fmt.Fprint(w, `{"event_id":"e"}`)
default:
fmt.Fprint(w, "data: {\"msg\":\"process_completed\",\"success\":false,\"output\":{\"error\":\"GPU занят\"}}\n\n")
}
}))
defer srv.Close()
if _, err := collect(t, aiConfig{Type: "gradio", URL: srv.URL}); err == nil || !strings.Contains(err.Error(), "GPU занят") {
t.Fatalf("err: %v", err)
}
}
func TestAIOpenAI(t *testing.T) {
var body map[string]any
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/v1/chat/completions" || r.Header.Get("Authorization") != "Bearer k" {
http.Error(w, "bad", 400)
return
}
_ = json.NewDecoder(r.Body).Decode(&body)
fmt.Fprint(w, "data: {\"choices\":[{\"delta\":{\"role\":\"assistant\"}}]}\n\ndata: {\"choices\":[{\"delta\":{\"content\":\"При\"}}]}\n\ndata: {\"choices\":[{\"delta\":{\"content\":\"вет\"}}]}\n\ndata: [DONE]\n\n")
}))
defer srv.Close()
got, err := collect(t, aiConfig{Type: "openai", URL: srv.URL + "/v1", Key: "k", Model: "m"})
if err != nil || got != "Привет" {
t.Fatalf("got %q, %v", got, err)
}
msgs := body["messages"].([]any)
if body["model"] != "m" || body["stream"] != true || len(msgs) != 4 || msgs[0].(map[string]any)["role"] != "system" {
t.Fatalf("body: %v", body)
}
}
func TestAIOpenAIAttachments(t *testing.T) {
var body map[string]any
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_ = json.NewDecoder(r.Body).Decode(&body)
fmt.Fprint(w, "data: [DONE]\n\n")
}))
defer srv.Close()
msg := aiMsg{Role: "user", Content: "посмотри", Attachments: []aiAttachment{
{Name: "image.png", MIME: "image/png", Data: "AQID"},
{Name: "voice.wav", MIME: "audio/wav", Data: "BAUG"},
{Name: "clip.mp4", MIME: "video/mp4", Data: "BwgJ"},
}}
if err := aiOpenAI(context.Background(), aiConfig{Type: "openai", URL: srv.URL, Model: "m"}, "sys", []aiMsg{msg}, func(string) {}); err != nil {
t.Fatal(err)
}
messages := body["messages"].([]any)
content := messages[1].(map[string]any)["content"].([]any)
if len(content) != 4 || content[3].(map[string]any)["text"] != "посмотри" {
t.Fatalf("content: %#v", content)
}
if got := content[0].(map[string]any)["image_url"].(map[string]any)["url"].(string); !strings.HasPrefix(got, "data:image/png;base64,") {
t.Fatalf("image: %q", got)
}
if got := content[1].(map[string]any)["input_audio"].(map[string]any)["format"]; got != "wav" {
t.Fatalf("audio: %#v", content[1])
}
video := content[2].(map[string]any)
if video["type"] != "video_url" || video["fps"] != float64(1) || !strings.HasPrefix(video["video_url"].(map[string]any)["url"].(string), "data:video/mp4;base64,") {
t.Fatalf("video: %#v", video)
}
}
func TestAIAnthropic(t *testing.T) {
var body map[string]any
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/v1/messages" || r.Header.Get("x-api-key") != "k" || r.Header.Get("anthropic-version") == "" {
http.Error(w, "bad", 400)
return
}
_ = json.NewDecoder(r.Body).Decode(&body)
fmt.Fprint(w, "event: message_start\ndata: {\"type\":\"message_start\"}\n\n"+
"event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"delta\":{\"type\":\"text_delta\",\"text\":\"При\"}}\n\n"+
"event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"delta\":{\"type\":\"text_delta\",\"text\":\"вет\"}}\n\n"+
"event: message_stop\ndata: {\"type\":\"message_stop\"}\n\n")
}))
defer srv.Close()
got, err := collect(t, aiConfig{Type: "anthropic", URL: srv.URL, Key: "k", Model: "m"})
if err != nil || got != "Привет" {
t.Fatalf("got %q, %v", got, err)
}
if body["system"] != "sys" || len(body["messages"].([]any)) != 3 {
t.Fatalf("body: %v", body)
}
}
// The chat is on by default, and an explicit user preference can still turn it off.
func TestAIChatDefaultOnAndCanBeDisabled(t *testing.T) {
s, c := newTestServer(t)
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Fprint(w, "data: {\"choices\":[{\"delta\":{\"content\":\"ok\"}}]}\n\ndata: [DONE]\n\n")
}))
defer srv.Close()
_ = s.db.SetConfig(cfgAIType, "openai")
_ = s.db.SetConfig(cfgAIURL, srv.URL)
post := func() *httptest.ResponseRecorder {
req := httptest.NewRequest(http.MethodPost, "/api/panel/ai/chat", strings.NewReader(`{"message":"hi","context":"x"}`))
req.AddCookie(c)
w := httptest.NewRecorder()
s.Handler().ServeHTTP(w, req)
return w
}
a, _ := s.db.AdminByUsername("adm")
state := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodGet, "/api/panel/ai", nil)
req.AddCookie(c)
s.handleAIState(state, req)
var got struct {
On bool `json:"on"`
}
if err := json.Unmarshal(state.Body.Bytes(), &got); err != nil || !got.On {
t.Fatalf("default state: %s (%v)", state.Body.String(), err)
}
w := post()
b, _ := io.ReadAll(w.Body)
if w.Code != 200 || string(b) != "ok" {
t.Fatalf("on: %d %q", w.Code, b)
}
// The talk is kept: the question and the answer.
cs := s.aiChats(a.ID)
// Named by the question at once (the model may rename it a moment later).
if len(cs) != 1 || cs[0].Title == "" || w.Header().Get("X-Chat-Id") != strconv.FormatInt(cs[0].ID, 10) {
t.Fatalf("chats: %+v %q", cs, w.Header().Get("X-Chat-Id"))
}
if h := s.aiHistoryOf(a.ID, cs[0].ID, 10); len(h) != 2 || h[0].Content != "hi" || h[1].Content != "ok" {
t.Fatalf("history: %+v", h)
}
off := httptest.NewRecorder()
req = httptest.NewRequest(http.MethodPost, "/api/panel/ai/toggle", strings.NewReader(`{"on":false}`))
req.AddCookie(c)
s.handleAIToggle(off, req)
state = httptest.NewRecorder()
req = httptest.NewRequest(http.MethodGet, "/api/panel/ai", nil)
req.AddCookie(c)
s.handleAIState(state, req)
var after struct {
On bool `json:"on"`
}
if err := json.Unmarshal(state.Body.Bytes(), &after); err != nil || after.On {
t.Fatalf("disabled state: %s (%v)", state.Body.String(), err)
}
if w := post(); w.Code != http.StatusForbidden {
t.Fatalf("explicit off: %d %s", w.Code, w.Body.String())
}
}
func TestAIChatKeepsAttachmentTextOnly(t *testing.T) {
s, c := newTestServer(t)
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
var in struct {
Messages []map[string]any `json:"messages"`
}
_ = json.NewDecoder(r.Body).Decode(&in)
content := in.Messages[len(in.Messages)-1]["content"].([]any)
if content[0].(map[string]any)["type"] != "image_url" {
http.Error(w, "image missing", http.StatusBadRequest)
return
}
fmt.Fprint(w, "data: [DONE]\n\n")
}))
defer srv.Close()
_ = s.db.SetConfig(cfgAIType, "openai")
_ = s.db.SetConfig(cfgAIURL, srv.URL)
a, _ := s.db.AdminByUsername("adm")
_ = s.db.SetConfig(aiUserKey(a.ID), "1")
chatID := s.aiNewChat(a.ID)
s.aiAppendTo(a.ID, chatID, "user", "предыдущий вопрос")
s.aiAppendTo(a.ID, chatID, "assistant", "ответ")
body := fmt.Sprintf(`{"message":"посмотри","chat":%d,"attachments":[{"name":"x.png","mime":"image/png","data":"AQID"}]}`, chatID)
req := httptest.NewRequest(http.MethodPost, "/api/panel/ai/chat", strings.NewReader(body))
req.AddCookie(c)
w := httptest.NewRecorder()
s.Handler().ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("response: %d %s", w.Code, w.Body.String())
}
chat, _ := strconv.ParseInt(w.Header().Get("X-Chat-Id"), 10, 64)
history := s.aiHistoryOf(a.ID, chat, 10)
if chat != chatID || len(history) != 3 || !strings.Contains(history[2].Content, "x.png") || strings.Contains(history[2].Content, "AQID") {
t.Fatalf("history contains wrong attachment data: %+v", history)
}
}
func TestVersionAtLeast(t *testing.T) {
for _, c := range []struct {
v string
want bool
}{{"0.2.0", true}, {"0.10.1", true}, {"0.1.9", false}, {"", false}, {"dev", false}, {"1.0", true}} {
if got := versionAtLeast(c.v, "0.2.0"); got != c.want {
t.Errorf("%q: %v", c.v, got)
}
}
}
// The window the model sees: newest first within the budget, old results
// and answers cut down, results as the user's words, same roles merged.
func TestAIWindow(t *testing.T) {
big := strings.Repeat("x", 5000)
rows := []aiMsg{
{Role: "user", Content: "старое"},
{Role: "assistant", Content: big},
{Role: "result", Content: big},
{Role: "user", Content: "дальше"},
{Role: "assistant", Content: "ок"},
{Role: "result", Content: "вывод"},
}
w := aiWindow(rows, 100000)
if len(w) != 5 || w[0].Role != "user" || w[len(w)-1].Role != "user" {
t.Fatalf("roles: %+v", w)
}
if len(w[1].Content) > 3200 || len(w[2].Content) > 1700+len("дальше")+40 {
t.Fatalf("old not cut: %d %d", len(w[1].Content), len(w[2].Content))
}
if !strings.Contains(w[4].Content, "вывод") || !strings.HasPrefix(w[4].Content, "Результат") {
t.Fatalf("last result: %q", w[4].Content)
}
// A small budget keeps the newest, starting with the user.
w = aiWindow(rows, 60)
if len(w) == 0 || w[0].Role != "user" || !strings.Contains(w[len(w)-1].Content, "вывод") {
t.Fatalf("small: %+v", w)
}
}
func TestStripThink(t *testing.T) {
if got := stripThink("::думаю::\nхм\n::ответ::\n391"); got != "391" {
t.Fatalf("%q", got)
}
if got := stripThink("просто"); got != "просто" {
t.Fatalf("%q", got)
}
}
// Several talks: each its own messages; a new one is named by its first
// question; deleting one leaves the others.
func TestAIChats(t *testing.T) {
s, _ := newTestServer(t)
a, _ := s.db.AdminByUsername("adm")
one := s.aiNewChat(a.ID)
two := s.aiNewChat(a.ID)
s.aiAppendTo(a.ID, one, "user", "первый вопрос про nginx и его настройки, длинный-длинный")
s.aiAppendTo(a.ID, two, "user", "второй")
if h := s.aiHistoryOf(a.ID, one, 10); len(h) != 1 || h[0].Content[:5] != "перв"[:5] {
t.Fatalf("one: %+v", h)
}
list := s.aiChats(a.ID)
if len(list) != 2 || list[0].ID != two || !strings.HasPrefix(list[1].Title, "первый вопрос") || len([]rune(list[1].Title)) > 41 {
t.Fatalf("list: %+v", list)
}
s.aiDeleteChat(a.ID, one)
if list = s.aiChats(a.ID); len(list) != 1 || len(s.aiHistoryOf(a.ID, one, 10)) != 0 {
t.Fatalf("after delete: %+v", list)
}
// Another user sees none of them and cannot write into them.
if len(s.aiChats(a.ID+1)) != 0 {
t.Fatal("another user sees the talks")
}
s.aiAppendTo(a.ID+1, two, "user", "чужое")
if len(s.aiHistoryOf(a.ID, two, 10)) != 1 {
t.Fatal("another user wrote into the talk")
}
}
// The thinking is kept with the answer (and how long it took), so it shows
// again later; the model never gets it back.
func TestAIThinkingKept(t *testing.T) {
s, c := newTestServer(t)
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fl := w.(http.Flusher)
fmt.Fprint(w, "data: {\"choices\":[{\"delta\":{\"content\":\"::думаю::\\nхм\\n\"}}]}\n\n")
fl.Flush()
time.Sleep(1100 * time.Millisecond)
fmt.Fprint(w, "data: {\"choices\":[{\"delta\":{\"content\":\"::ответ::\\n391\"}}]}\n\ndata: [DONE]\n\n")
}))
defer srv.Close()
_ = s.db.SetConfig(cfgAIType, "openai")
_ = s.db.SetConfig(cfgAIURL, srv.URL)
a, _ := s.db.AdminByUsername("adm")
_ = s.db.SetConfig(aiUserKey(a.ID), "1")
req := httptest.NewRequest(http.MethodPost, "/api/panel/ai/chat", strings.NewReader(`{"message":"17*23","think":true}`))
req.AddCookie(c)
w := httptest.NewRecorder()
s.Handler().ServeHTTP(w, req)
chat, _ := strconv.ParseInt(w.Header().Get("X-Chat-Id"), 10, 64)
h := s.aiHistoryOf(a.ID, chat, 10)
if len(h) != 2 || !strings.Contains(h[1].Content, "::думаю::") || h[1].ThinkMS < 1000 {
t.Fatalf("kept: %+v", h)
}
if win := aiWindow(h, 10000); strings.Contains(win[len(win)-1].Content, "хм") {
t.Fatalf("the model gets the thinking back: %+v", win)
}
}
// A new chat is named by the model: its first question in two or three
// words, asked aside so the answer does not wait.
func TestAIChatTitle(t *testing.T) {
s, c := newTestServer(t)
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
var in struct {
Messages []aiMsg `json:"messages"`
}
_ = json.NewDecoder(r.Body).Decode(&in)
reply := "ok"
if strings.Contains(in.Messages[0].Content, "название") {
reply = "«Выбор поисковика.»"
}
b, _ := json.Marshal(reply)
fmt.Fprintf(w, "data: {\"choices\":[{\"delta\":{\"content\":%s}}]}\n\ndata: [DONE]\n\n", b)
}))
defer srv.Close()
_ = s.db.SetConfig(cfgAIType, "openai")
_ = s.db.SetConfig(cfgAIURL, srv.URL)
a, _ := s.db.AdminByUsername("adm")
_ = s.db.SetConfig(aiUserKey(a.ID), "1")
req := httptest.NewRequest(http.MethodPost, "/api/panel/ai/chat", strings.NewReader(`{"message":"какой поисковик ты используешь сейчас"}`))
req.AddCookie(c)
w := httptest.NewRecorder()
s.Handler().ServeHTTP(w, req)
for i := 0; i < 50; i++ {
if cs := s.aiChats(a.ID); len(cs) == 1 && cs[0].Title == "Выбор поисковика" {
return
}
time.Sleep(50 * time.Millisecond)
}
t.Fatalf("title: %+v", s.aiChats(a.ID))
}
// Qwen Omni's files go where its inputs are (audio before image there), and
// a video as Gradio 5 takes one: {video, subtitles}.
func TestAIGradioMediaSlots(t *testing.T) {
var joined map[string]any
n := 0
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/config":
fmt.Fprint(w, `{"components":[{"id":20,"type":"textbox"},{"id":16,"type":"audio"},{"id":17,"type":"image"},{"id":18,"type":"video"},`+
`{"id":13,"type":"chatbot"},{"id":7,"type":"textbox"},{"id":8,"type":"slider"},{"id":9,"type":"slider"},{"id":10,"type":"slider"}],`+
`"dependencies":[{"id":0,"api_name":"chat_predict","targets":[[21,"click"]],"inputs":[20,16,17,18,13,7,8,9,10]}]}`)
case "/gradio_api/upload":
n++
fmt.Fprintf(w, `["/tmp/f%d"]`, n)
case "/gradio_api/queue/join":
_ = json.NewDecoder(r.Body).Decode(&joined)
fmt.Fprint(w, `{"event_id":"e"}`)
case "/gradio_api/queue/data":
fmt.Fprint(w, "data: {\"msg\":\"process_completed\",\"success\":true,\"output\":{\"data\":[{},{},{},{},[{\"role\":\"assistant\",\"content\":\"ok\"}],{},{}]}}\n\n")
}
}))
defer srv.Close()
enc := base64.StdEncoding.EncodeToString([]byte("x"))
msgs := []aiMsg{{Role: "user", Content: "что тут", Attachments: []aiAttachment{
{Name: "a.png", MIME: "image/png", Data: enc},
{Name: "b.mp3", MIME: "audio/mpeg", Data: enc},
{Name: "c.mp4", MIME: "video/mp4", Data: enc},
}}}
if err := aiStream(context.Background(), aiConfig{Type: "gradio", URL: srv.URL}, "sys", msgs, func(string) {}); err != nil {
t.Fatal(err)
}
d := joined["data"].([]any)
path := func(v any) string { m, _ := v.(map[string]any); p, _ := m["path"].(string); return p }
video, _ := d[3].(map[string]any)
if path(d[1]) != "/tmp/f2" || path(d[2]) != "/tmp/f1" || path(video["video"]) != "/tmp/f3" || video["subtitles"] != nil || d[5] != "sys" {
t.Fatalf("data: %v", d)
}
}
+110
View File
@@ -0,0 +1,110 @@
package server
import (
"encoding/json"
"fmt"
"net/http"
"net/http/httptest"
"net/url"
"strings"
"testing"
)
// The model set before there could be several is the first of the list.
func TestAIModelsFromLegacy(t *testing.T) {
s, _ := newTestServer(t)
_ = s.db.SetConfig(cfgAIType, "openai")
_ = s.db.SetConfig(cfgAIURL, "https://api.example/v1/")
_ = s.db.SetConfig(cfgAIModel, "gpt-x")
_ = s.db.SetConfig(cfgAIKey, "k1")
ms := s.aiModels()
if len(ms) != 1 || ms[0].Type != "openai" || ms[0].URL != "https://api.example/v1" || ms[0].Model != "gpt-x" || ms[0].Key != "k1" || ms[0].ID == "" {
t.Fatalf("models: %+v", ms)
}
if got := s.aiModel("nope"); got.ID != ms[0].ID {
t.Fatalf("an unknown id is not the first model: %+v", got)
}
}
// postAIModels sends the settings form with the given models.
func postAIModels(t *testing.T, s *Server, c *http.Cookie, models [][7]string) {
t.Helper()
f := url.Values{"csrf": {"tok"}}
for _, m := range models {
f.Add("m_id", m[0])
f.Add("m_name", m[1])
f.Add("m_type", m[2])
f.Add("m_url", m[3])
f.Add("m_model", m[4])
f.Add("m_key", m[5])
f.Add("m_short", m[6])
}
req := httptest.NewRequest(http.MethodPost, "/settings/ai", strings.NewReader(f.Encode()))
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
req.AddCookie(c)
req.AddCookie(&http.Cookie{Name: csrfCookie, Value: "tok"})
w := httptest.NewRecorder()
s.Handler().ServeHTTP(w, req)
if w.Code != http.StatusSeeOther {
t.Fatalf("save: %d %s", w.Code, w.Body)
}
}
// Настройки keep several models with their names; the chat learns the
// names and ids only, never a key.
func TestAIModelsSaveAndList(t *testing.T) {
s, c := newTestServer(t)
postAIModels(t, s, c, [][7]string{
{"", "Qwen", "gradio", "", "", "", ""},
{"", "Claude", "anthropic", "https://api.anthropic.com", "claude-x", "secret-key", "Сонет 4.5 длинно"},
{"", "", "", "", "", "", ""}, // an empty card is dropped
})
ms := s.aiModels()
if len(ms) != 2 || ms[0].Name != "Qwen" || ms[1].Name != "Claude" || ms[1].Key != "secret-key" || ms[0].ID == "" || ms[0].ID == ms[1].ID || ms[1].Short != "Сонет 4.5 дл" {
t.Fatalf("models: %+v", ms)
}
body := get(t, s, c, "/api/panel/ai").Body.String()
if strings.Contains(body, "secret-key") || !strings.Contains(body, `"name":"Claude"`) || !strings.Contains(body, `"short":"Сонет 4.5 дл"`) || !strings.Contains(body, ms[1].ID) {
t.Fatalf("state: %s", body)
}
// Saved again with its id, a model keeps it.
postAIModels(t, s, c, [][7]string{{ms[1].ID, "Claude 2", "anthropic", "https://api.anthropic.com", "claude-x", "secret-key", ""}})
if got := s.aiModels(); len(got) != 1 || got[0].ID != ms[1].ID || got[0].Name != "Claude 2" {
t.Fatalf("after resave: %+v", got)
}
if !strings.Contains(get(t, s, c, "/settings?frag=1").Body.String(), `value="Claude 2"`) {
t.Fatal("the settings page does not show the model's name")
}
}
// The chat asks the model it names.
func TestAIChatUsesChosenModel(t *testing.T) {
s, c := newTestServer(t)
srv := func(word string) *httptest.Server {
return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "data: {\"choices\":[{\"delta\":{\"content\":\"%s\"}}]}\n\ndata: [DONE]\n\n", word)
}))
}
one, two := srv("first"), srv("second")
defer one.Close()
defer two.Close()
postAIModels(t, s, c, [][7]string{
{"", "One", "openai", one.URL, "m", "", ""},
{"", "Two", "openai", two.URL, "m", "", ""},
})
ms := s.aiModels()
ask := func(model string) string {
b, _ := json.Marshal(map[string]any{"message": "hi", "context": "x", "model": model})
req := httptest.NewRequest(http.MethodPost, "/api/panel/ai/chat", strings.NewReader(string(b)))
req.AddCookie(c)
w := httptest.NewRecorder()
s.Handler().ServeHTTP(w, req)
return w.Body.String()
}
if got := ask(ms[1].ID); !strings.Contains(got, "second") {
t.Fatalf("model Two: %q", got)
}
if got := ask(""); !strings.Contains(got, "first") {
t.Fatalf("no model named - the first: %q", got)
}
}
+42
View File
@@ -0,0 +1,42 @@
package server
import (
"fmt"
"net/http"
"net/http/httptest"
"strings"
"testing"
)
// The assistant is on unless the user turned it off - for the web search
// and page reading as for the chat (a user who never touched the switch
// has no setting at all).
func TestAISearchAndFetchOnByDefault(t *testing.T) {
s, c := newTestServer(t)
sx := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Fprint(w, `{"results":[{"title":"Intel","url":"https://example.com/intel","content":"процессоры"}]}`)
}))
defer sx.Close()
_ = s.db.SetConfig(cfgSearchEngine, "searxng")
_ = s.db.SetConfig(cfgSearchURL, sx.URL)
post := func(path, body string) string {
req := httptest.NewRequest(http.MethodPost, path, strings.NewReader(body))
req.AddCookie(c)
w := httptest.NewRecorder()
s.Handler().ServeHTTP(w, req)
return w.Body.String()
}
if got := post("/api/panel/ai/search", `{"q":"intel"}`); !strings.Contains(got, `"ok":true`) {
t.Fatalf("search by default: %s", got)
}
if got := post("/api/panel/ai/fetch", `{"url":"не адрес"}`); strings.Contains(got, "выключен") {
t.Fatalf("fetch by default: %s", got)
}
a, _ := s.db.AdminByUsername("adm")
_ = s.db.SetConfig(aiUserKey(a.ID), "0")
for _, p := range []string{"/api/panel/ai/search", "/api/panel/ai/fetch"} {
if got := post(p, `{"q":"intel","url":"https://example.com"}`); !strings.Contains(got, "выключен") {
t.Fatalf("%s with the assistant off: %s", p, got)
}
}
}
+144
View File
@@ -0,0 +1,144 @@
package server
import (
"encoding/json"
"fmt"
"net/http"
"net/http/httptest"
"strings"
"sync"
"testing"
)
// openAIChunk is one streamed delta of an OpenAI-compatible server.
func openAIChunk(field, text string) string {
b, _ := json.Marshal(map[string]any{"choices": []any{map[string]any{"delta": map[string]string{field: text}}}})
return "data: " + string(b) + "\n\n"
}
// A model that thinks by itself (llama.cpp, vLLM: reasoning_content) has its
// thinking shown as the chat's own: between ::думаю:: and ::ответ::.
func TestAIOpenAINativeReasoning(t *testing.T) {
var body map[string]any
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_ = json.NewDecoder(r.Body).Decode(&body)
fmt.Fprint(w, openAIChunk("reasoning_content", "17*23")+openAIChunk("reasoning_content", " = 391")+
openAIChunk("content", "391")+"data: [DONE]\n\n")
}))
defer srv.Close()
got, err := collect(t, aiConfig{Type: "openai", URL: srv.URL, Model: "m", think: "on"})
if err != nil || got != "::думаю::\n17*23 = 391\n::ответ::\n391" {
t.Fatalf("got %q, %v", got, err)
}
kw, _ := body["chat_template_kwargs"].(map[string]any)
if kw["enable_thinking"] != true {
t.Fatalf("thinking not asked for: %v", body)
}
}
// A server that does not know the thinking switch (400 on it) is asked again
// without it.
func TestAIOpenAIRetriesWithoutThinkSwitch(t *testing.T) {
var mu sync.Mutex
var calls []map[string]any
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
var b map[string]any
_ = json.NewDecoder(r.Body).Decode(&b)
mu.Lock()
calls = append(calls, b)
mu.Unlock()
if _, ok := b["chat_template_kwargs"]; ok {
http.Error(w, `{"error":{"message":"Unrecognized request argument supplied: chat_template_kwargs"}}`, 400)
return
}
fmt.Fprint(w, openAIChunk("content", "ok")+"data: [DONE]\n\n")
}))
defer srv.Close()
got, err := collect(t, aiConfig{Type: "openai", URL: srv.URL, Model: "m", think: "off"})
if err != nil || got != "ok" || len(calls) != 2 {
t.Fatalf("got %q, %v, %d calls", got, err, len(calls))
}
}
// Anthropic: «Думать» turns on extended thinking; its thinking_delta is the
// thinking shown.
func TestAIAnthropicThinking(t *testing.T) {
var body map[string]any
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_ = json.NewDecoder(r.Body).Decode(&body)
fmt.Fprint(w, "data: {\"type\":\"content_block_delta\",\"delta\":{\"type\":\"thinking_delta\",\"thinking\":\"хм\"}}\n\n"+
"data: {\"type\":\"content_block_delta\",\"delta\":{\"type\":\"text_delta\",\"text\":\"Да\"}}\n\n"+
"data: {\"type\":\"message_stop\"}\n\n")
}))
defer srv.Close()
got, err := collect(t, aiConfig{Type: "anthropic", URL: srv.URL, Key: "k", Model: "m", think: "on"})
if err != nil || got != "::думаю::\nхм\n::ответ::\nДа" {
t.Fatalf("got %q, %v", got, err)
}
th, _ := body["thinking"].(map[string]any)
if th["type"] != "enabled" || body["max_tokens"].(float64) <= th["budget_tokens"].(float64) {
t.Fatalf("thinking not asked for: %v", body)
}
}
// askThink sends one «Думать» question to the chat and gives the answer and
// the system prompts the model got, in order.
func askThink(t *testing.T, reply func(system string, body map[string]any) string) (string, []string) {
t.Helper()
s, c := newTestServer(t)
var mu sync.Mutex
var systems []string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
var b map[string]any
_ = json.NewDecoder(r.Body).Decode(&b)
sys := b["messages"].([]any)[0].(map[string]any)["content"].(string)
if strings.Contains(fmt.Sprint(b["messages"]), "название чата") || strings.Contains(sys, "название чата") {
fmt.Fprint(w, openAIChunk("content", "Чат")+"data: [DONE]\n\n")
return
}
mu.Lock()
systems = append(systems, sys)
mu.Unlock()
fmt.Fprint(w, reply(sys, b)+"data: [DONE]\n\n")
}))
t.Cleanup(srv.Close)
postAIModels(t, s, c, [][7]string{{"", "M", "openai", srv.URL, "m", "", ""}})
req := httptest.NewRequest(http.MethodPost, "/api/panel/ai/chat", strings.NewReader(`{"message":"сколько 17*23","context":"x","think":true}`))
req.AddCookie(c)
w := httptest.NewRecorder()
s.Handler().ServeHTTP(w, req)
mu.Lock()
defer mu.Unlock()
return w.Body.String(), append([]string(nil), systems...)
}
// A model with thinking of its own thinks once, by itself: no "think in
// words" prompt on top.
func TestAIChatThinkNative(t *testing.T) {
got, systems := askThink(t, func(sys string, b map[string]any) string {
return openAIChunk("reasoning_content", "считаю") + openAIChunk("content", "391")
})
if len(systems) != 1 || strings.Contains(systems[0], "Режим размышления") {
t.Fatalf("systems: %q", systems)
}
if !strings.Contains(got, "::думаю::\nсчитаю\n::ответ::\n391") {
t.Fatalf("answer: %q", got)
}
}
// A model without it is asked again with the prompt, its first answer
// dropped unseen.
func TestAIChatThinkPromptFallback(t *testing.T) {
got, systems := askThink(t, func(sys string, b map[string]any) string {
if strings.Contains(sys, "Режим размышления") {
return openAIChunk("content", "::думаю::\nсчитаю\n::ответ::\n391")
}
return openAIChunk("content", "сразу ответ")
})
if len(systems) != 2 || strings.Contains(systems[0], "Режим размышления") || !strings.Contains(systems[1], "Режим размышления") {
t.Fatalf("systems: %q", systems)
}
if strings.Contains(got, "сразу ответ") || !strings.Contains(got, "::ответ::\n391") {
t.Fatalf("answer: %q", got)
}
}
+89
View File
@@ -0,0 +1,89 @@
package server
import (
"bytes"
"os"
"path/filepath"
"testing"
)
// A chat deleted takes with it what the user attached to it: the text of
// its files lives in the messages, and must be gone from the database
// files too - not just from the tables (SQLite leaves deleted rows in its
// free pages and in the write-ahead log).
func TestDeleteChatWipesAttachments(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "t.db")
db, err := OpenDB(path)
if err != nil {
t.Fatal(err)
}
s, err := New(Options{DB: db, DistDir: t.TempDir()})
if err != nil {
t.Fatal(err)
}
hash, _ := hashPassword("Passw0rd12345")
if err := db.CreateAdmin("adm", hash); err != nil {
t.Fatal(err)
}
adm, _ := db.AdminByUsername("adm")
me := adm.ID
const secret = "TOP-SECRET-CONTENT-OF-AN-ATTACHED-FILE-7731"
chat := s.aiNewChat(me)
other := s.aiNewChat(me)
s.aiAppendTo(me, chat, "user", "посмотри\n\n[Вложение: notes.txt]\nСодержимое файла:\n"+secret)
s.aiAppendTo(me, chat, "result", "Картинка готова [картинка: https://s2.x-grk.com/uploads/userphoto/ai/zzz-7731.png]")
s.aiAppendTo(me, other, "user", "другой чат, остаётся")
s.aiDeleteChat(me, chat)
var n int
_ = db.sql.QueryRow(`SELECT COUNT(*) FROM ai_messages WHERE chat_id = ?`, chat).Scan(&n)
if n != 0 {
t.Fatalf("%d messages of the deleted chat remain", n)
}
_ = db.sql.QueryRow(`SELECT COUNT(*) FROM ai_messages WHERE chat_id = ?`, other).Scan(&n)
if n != 1 {
t.Fatalf("the other chat has %d messages, want 1", n)
}
for _, f := range []string{path, path + "-wal"} {
b, _ := os.ReadFile(f)
if bytes.Contains(b, []byte(secret)) || bytes.Contains(b, []byte("zzz-7731")) {
t.Fatalf("the deleted chat's content is still in %s", filepath.Base(f))
}
}
}
// The oldest chats go past the limit the same way: nothing of them is kept.
func TestOldChatsWipedToo(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "t.db")
db, err := OpenDB(path)
if err != nil {
t.Fatal(err)
}
s, err := New(Options{DB: db, DistDir: t.TempDir()})
if err != nil {
t.Fatal(err)
}
hash, _ := hashPassword("Passw0rd12345")
if err := db.CreateAdmin("adm", hash); err != nil {
t.Fatal(err)
}
adm, _ := db.AdminByUsername("adm")
me := adm.ID
first := s.aiNewChat(me)
s.aiAppendTo(me, first, "user", "[Вложение: a.txt]\nСодержимое файла:\nOLD-ATTACHMENT-TEXT-5512")
for i := 0; i < aiChatsKeep; i++ {
s.aiNewChat(me)
}
if s.aiOwns(me, first) {
t.Fatal("the oldest chat is kept past the limit")
}
for _, f := range []string{path, path + "-wal"} {
b, _ := os.ReadFile(f)
if bytes.Contains(b, []byte("OLD-ATTACHMENT-TEXT-5512")) {
t.Fatalf("the old chat's content is still in %s", filepath.Base(f))
}
}
}
+75 -12
View File
@@ -3,6 +3,10 @@ package server
import (
"crypto/rand"
"encoding/hex"
"net/url"
"strconv"
lpra "github.com/x-grk/longpolling-remote-access"
)
// Config keys stored in the config table.
@@ -11,16 +15,20 @@ const (
cfgEnrollSecret = "enroll_secret"
cfgSetupDone = "setup_done"
cfgInstallCode = "install_code"
cfgInstallLen = "install_code_len"
cfgUfwEnabled = "ufw_enabled"
cfgTunnelHost = "tunnel_host"
cfgTunnelPort = "tunnel_port"
cfgDirectAddr = "direct_address"
)
// DefaultHostAddress is the public URL agents dial by default. It is baked into
// the rendered install.sh and can be changed from the settings page.
const DefaultHostAddress = "https://REDACTED:8445"
// Version is the server build version, injected at compile time via -ldflags
// (-X .../internal/server.Version=...). It is the newest version agents update to.
var Version = "dev"
// Version is the server build version (see VERSION.txt). It is the newest
// version agents update to.
var Version = lpra.Version
// ensureDefaults seeds the host address and a random enroll secret on first boot.
func (db *DB) ensureDefaults() error {
@@ -47,7 +55,7 @@ func (db *DB) ensureDefaults() error {
return err
}
if code == "" {
if err := db.SetConfig(cfgInstallCode, shortCode()); err != nil {
if err := db.SetConfig(cfgInstallCode, shortCode(db.InstallCodeLen())); err != nil {
return err
}
}
@@ -62,6 +70,23 @@ func (db *DB) HostAddress() string {
return v
}
// DirectAddress is the server's direct URL, one that is not behind a proxy
// like Cloudflare: the agents' fallback, and a way to install when the main
// address is out of reach. Empty when not set.
func (db *DB) DirectAddress() string {
v, _ := db.GetConfig(cfgDirectAddr)
return v
}
// directHost is the direct address's host name, without scheme and port.
func directHost(addr string) string {
u, err := url.Parse(addr)
if err != nil {
return ""
}
return u.Hostname()
}
func (db *DB) EnrollSecret() string {
v, _ := db.GetConfig(cfgEnrollSecret)
return v
@@ -86,6 +111,24 @@ func (db *DB) SetUfwEnabled(on bool) error {
return db.SetConfig(cfgUfwEnabled, "0")
}
// TunnelHost is the host agents dial for the direct tunnel. Empty means the
// host of the URL they already reach the panel on.
func (db *DB) TunnelHost() string {
v, _ := db.GetConfig(cfgTunnelHost)
return v
}
// TunnelPort is the direct-tunnel port set in Settings, or 0 when it is left
// at the port of the server's -tunnel address.
func (db *DB) TunnelPort() int {
v, _ := db.GetConfig(cfgTunnelPort)
n, _ := strconv.Atoi(v)
if n < 1 || n > 65535 {
return 0
}
return n
}
func (db *DB) SetupDone() bool {
v, _ := db.GetConfig(cfgSetupDone)
return v == "1"
@@ -102,16 +145,36 @@ func randToken() string {
return hex.EncodeToString(b)
}
// shortCode returns a short, URL-safe, typo-resistant code for install links.
// It avoids ambiguous characters (0/O, 1/l/I) so it is easy to read and type.
func shortCode() string {
const alphabet = "abcdefghjkmnpqrstuvwxyz23456789"
b := make([]byte, 7)
// Install code length: 3 by default, up to 24.
const (
installCodeMin = 3
installCodeMax = 24
installCodeDef = 3
)
// InstallCodeLen is how many characters a new install code has.
func (db *DB) InstallCodeLen() int {
v, _ := db.GetConfig(cfgInstallLen)
if n, err := strconv.Atoi(v); err == nil && n >= installCodeMin && n <= installCodeMax {
return n
}
return installCodeDef
}
// shortCode returns a random code of n English letters and digits for the
// /i/<code> install link. Every character is equally likely.
func shortCode(n int) string {
const alphabet = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"
out := make([]byte, 0, n)
b := make([]byte, 1)
for len(out) < n {
if _, err := rand.Read(b); err != nil {
panic("crypto/rand failed: " + err.Error())
}
for i := range b {
b[i] = alphabet[int(b[i])%len(alphabet)]
// 248 = 4 * 62: drop the rest, so no character comes up more often.
if b[0] < 248 {
out = append(out, alphabet[int(b[0])%len(alphabet)])
}
return string(b)
}
return string(out)
}
+334 -37
View File
@@ -11,7 +11,8 @@ import (
// DB wraps the SQLite connection and all queries used by the server.
type DB struct {
sql *sql.DB
sql *dbx
path string // the file; the key of saved secrets sits beside it
}
// OpenDB opens (creating if needed) the SQLite database at path and runs
@@ -19,7 +20,7 @@ type DB struct {
func OpenDB(path string) (*DB, error) {
// _pragma busy_timeout avoids "database is locked" under the concurrent
// long-poll load; foreign_keys keeps the schema honest.
dsn := fmt.Sprintf("file:%s?_pragma=busy_timeout(5000)&_pragma=foreign_keys(1)&_pragma=journal_mode(WAL)", path)
dsn := fmt.Sprintf("file:%s?_pragma=busy_timeout(5000)&_pragma=foreign_keys(1)&_pragma=journal_mode(WAL)&_pragma=secure_delete(1)", path)
sdb, err := sql.Open("sqlite", dsn)
if err != nil {
return nil, err
@@ -30,7 +31,7 @@ func OpenDB(path string) (*DB, error) {
if err := sdb.Ping(); err != nil {
return nil, err
}
db := &DB{sql: sdb}
db := &DB{sql: &dbx{DB: sdb}, path: path}
if err := db.migrate(); err != nil {
return nil, err
}
@@ -85,6 +86,16 @@ CREATE TABLE IF NOT EXISTS ufw_rules (
created_at INTEGER NOT NULL,
UNIQUE(port, proto)
);
CREATE TABLE IF NOT EXISTS admin_keys (
id INTEGER PRIMARY KEY AUTOINCREMENT,
admin_id INTEGER NOT NULL REFERENCES admins(id) ON DELETE CASCADE,
cred_id BLOB UNIQUE NOT NULL,
public_key BLOB NOT NULL,
sign_count INTEGER NOT NULL DEFAULT 0,
name TEXT NOT NULL DEFAULT '',
created_at INTEGER NOT NULL,
last_used INTEGER NOT NULL DEFAULT 0
);
CREATE TABLE IF NOT EXISTS forward_rules (
id INTEGER PRIMARY KEY AUTOINCREMENT,
agent_id TEXT NOT NULL,
@@ -105,11 +116,43 @@ CREATE TABLE IF NOT EXISTS forward_rules (
`ALTER TABLE ufw_rules ADD COLUMN rule_id INTEGER NOT NULL DEFAULT 0`,
`ALTER TABLE agents ADD COLUMN run_user TEXT NOT NULL DEFAULT ''`,
`ALTER TABLE agents ADD COLUMN login_user TEXT NOT NULL DEFAULT ''`,
`ALTER TABLE forward_rules ADD COLUMN no_ufw INTEGER NOT NULL DEFAULT 0`,
`ALTER TABLE forward_rules ADD COLUMN label TEXT NOT NULL DEFAULT ''`,
`ALTER TABLE admins ADD COLUMN password_off INTEGER NOT NULL DEFAULT 0`,
`ALTER TABLE agents ADD COLUMN transport TEXT NOT NULL DEFAULT 'auto'`,
`ALTER TABLE admins ADD COLUMN pin_hash TEXT NOT NULL DEFAULT ''`,
`ALTER TABLE admins ADD COLUMN role TEXT NOT NULL DEFAULT 'admin'`,
`ALTER TABLE admins ADD COLUMN pw_locked INTEGER NOT NULL DEFAULT 0`,
`CREATE TABLE IF NOT EXISTS ai_messages (
id INTEGER PRIMARY KEY AUTOINCREMENT,
admin_id INTEGER NOT NULL REFERENCES admins(id) ON DELETE CASCADE,
role TEXT NOT NULL, content TEXT NOT NULL, at INTEGER NOT NULL)`,
`CREATE INDEX IF NOT EXISTS ai_messages_admin ON ai_messages(admin_id, id)`,
`ALTER TABLE ai_messages ADD COLUMN chat_id INTEGER NOT NULL DEFAULT 0`,
`ALTER TABLE ai_messages ADD COLUMN think_ms INTEGER NOT NULL DEFAULT 0`,
`CREATE TABLE IF NOT EXISTS ai_chats (
id INTEGER PRIMARY KEY AUTOINCREMENT,
admin_id INTEGER NOT NULL REFERENCES admins(id) ON DELETE CASCADE,
title TEXT NOT NULL, created INTEGER NOT NULL, updated INTEGER NOT NULL)`,
`CREATE INDEX IF NOT EXISTS ai_messages_chat ON ai_messages(chat_id, id)`,
`CREATE TABLE IF NOT EXISTS ssh_hosts (
id INTEGER PRIMARY KEY AUTOINCREMENT,
admin_id INTEGER NOT NULL REFERENCES admins(id) ON DELETE CASCADE,
label TEXT NOT NULL, host TEXT NOT NULL, port INTEGER NOT NULL, login TEXT NOT NULL,
auth TEXT NOT NULL, secret TEXT NOT NULL)`,
`CREATE TABLE IF NOT EXISTS admin_agents (
admin_id INTEGER NOT NULL REFERENCES admins(id) ON DELETE CASCADE,
agent_id TEXT NOT NULL,
PRIMARY KEY (admin_id, agent_id))`,
} {
if _, err := db.sql.Exec(alter); err != nil && !strings.Contains(err.Error(), "duplicate column") {
return err
}
}
// ssh_hosts.user (a word PostgreSQL keeps for itself) became login.
if !db.sql.pg {
_, _ = db.sql.Exec(`ALTER TABLE ssh_hosts RENAME COLUMN user TO login`)
}
return nil
}
@@ -140,24 +183,52 @@ func (db *DB) CountAdmins() (int, error) {
}
func (db *DB) CreateAdmin(username, passHash string) error {
_, err := db.sql.Exec(
`INSERT INTO admins(username, pass_hash, created_at) VALUES(?, ?, ?)`,
username, passHash, time.Now().Unix())
_, err := db.CreateUser(username, passHash, RoleAdmin)
return err
}
// Admin is one row of the admins table.
// CreateUser adds a panel user with the role given and returns its id.
func (db *DB) CreateUser(username, passHash, role string) (int64, error) {
return db.sql.insert(
`INSERT INTO admins(username, pass_hash, created_at, role) VALUES(?, ?, ?, ?)`,
username, passHash, time.Now().Unix(), role)
}
// Roles of panel users: an admin sees and runs everything; a user only the
// machines given to them, and none of the settings.
const (
RoleAdmin = "admin"
RoleUser = "user"
)
// Admin is one row of the admins table: a panel user, an admin or not.
type Admin struct {
ID int64
Username string
PassHash string
Role string
// PwLocked keeps the user from turning the password sign-in on or off
// themselves: an admin set it.
PwLocked bool
}
// IsAdmin reports whether the user has the run of the whole panel.
func (a *Admin) IsAdmin() bool { return a != nil && a.Role != RoleUser }
const adminCols = `id, username, pass_hash, role, pw_locked`
func scanAdmin(row interface{ Scan(...any) error }) (*Admin, error) {
a := &Admin{}
var locked int
if err := row.Scan(&a.ID, &a.Username, &a.PassHash, &a.Role, &locked); err != nil {
return nil, err
}
a.PwLocked = locked == 1
return a, nil
}
func (db *DB) AdminByUsername(username string) (*Admin, error) {
a := &Admin{}
err := db.sql.QueryRow(
`SELECT id, username, pass_hash FROM admins WHERE username = ?`, username).
Scan(&a.ID, &a.Username, &a.PassHash)
a, err := scanAdmin(db.sql.QueryRow(`SELECT `+adminCols+` FROM admins WHERE username = ?`, username))
if err == sql.ErrNoRows {
return nil, nil
}
@@ -165,27 +236,215 @@ func (db *DB) AdminByUsername(username string) (*Admin, error) {
}
func (db *DB) ListAdmins() ([]Admin, error) {
rows, err := db.sql.Query(`SELECT id, username FROM admins ORDER BY username`)
rows, err := db.sql.Query(`SELECT ` + adminCols + ` FROM admins ORDER BY role, username`)
if err != nil {
return nil, err
}
defer rows.Close()
var out []Admin
for rows.Next() {
var a Admin
if err := rows.Scan(&a.ID, &a.Username); err != nil {
a, err := scanAdmin(rows)
if err != nil {
return nil, err
}
out = append(out, a)
out = append(out, *a)
}
return out, rows.Err()
}
// CountRole is how many users have the role.
func (db *DB) CountRole(role string) int {
var n int
_ = db.sql.QueryRow(`SELECT COUNT(*) FROM admins WHERE role = ?`, role).Scan(&n)
return n
}
func (db *DB) SetUserRole(id int64, role string) error {
_, err := db.sql.Exec(`UPDATE admins SET role = ? WHERE id = ?`, role, id)
return err
}
func (db *DB) SetPwLocked(id int64, locked bool) error {
_, err := db.sql.Exec(`UPDATE admins SET pw_locked = ? WHERE id = ?`, boolInt(locked), id)
return err
}
// UserAgents are the machines a user may use.
func (db *DB) UserAgents(adminID int64) map[string]bool {
out := map[string]bool{}
rows, err := db.sql.Query(`SELECT agent_id FROM admin_agents WHERE admin_id = ?`, adminID)
if err != nil {
return out
}
defer rows.Close()
for rows.Next() {
var id string
if rows.Scan(&id) == nil {
out[id] = true
}
}
return out
}
// SetUserAgents gives the user exactly these machines.
func (db *DB) SetUserAgents(adminID int64, agentIDs []string) error {
tx, err := db.sql.Begin()
if err != nil {
return err
}
defer tx.Rollback()
if _, err := tx.Exec(`DELETE FROM admin_agents WHERE admin_id = ?`, adminID); err != nil {
return err
}
for _, id := range agentIDs {
if _, err := tx.Exec(`INSERT OR IGNORE INTO admin_agents(admin_id, agent_id) VALUES(?, ?)`, adminID, id); err != nil {
return err
}
}
return tx.Commit()
}
// CanUseAgent reports whether the user may use the machine.
func (db *DB) CanUseAgent(a *Admin, agentID string) bool {
if a == nil {
return false
}
if a.IsAdmin() {
return true
}
var n int
_ = db.sql.QueryRow(`SELECT COUNT(*) FROM admin_agents WHERE admin_id = ? AND agent_id = ?`, a.ID, agentID).Scan(&n)
return n > 0
}
func (db *DB) DeleteAdmin(id int64) error {
_, err := db.sql.Exec(`DELETE FROM admins WHERE id = ?`, id)
return err
}
// AdminKey is a passkey an admin signs in with.
type AdminKey struct {
ID int64
AdminID int64
CredID []byte
PublicKey []byte // PKIX DER
SignCount uint32
Name string
CreatedAt int64
LastUsed int64
}
func (db *DB) AdminByID(id int64) (*Admin, error) {
a, err := scanAdmin(db.sql.QueryRow(`SELECT `+adminCols+` FROM admins WHERE id = ?`, id))
if err == sql.ErrNoRows {
return nil, nil
}
return a, err
}
func (db *DB) AddAdminKey(k AdminKey) error {
_, err := db.sql.Exec(
`INSERT INTO admin_keys(admin_id, cred_id, public_key, sign_count, name, created_at) VALUES(?, ?, ?, ?, ?, ?)`,
k.AdminID, k.CredID, k.PublicKey, k.SignCount, k.Name, time.Now().Unix())
return err
}
func (db *DB) ListAdminKeys(adminID int64) ([]AdminKey, error) {
rows, err := db.sql.Query(
`SELECT id, admin_id, cred_id, public_key, sign_count, name, created_at, last_used
FROM admin_keys WHERE admin_id = ? ORDER BY id`, adminID)
if err != nil {
return nil, err
}
defer rows.Close()
var out []AdminKey
for rows.Next() {
var k AdminKey
if err := rows.Scan(&k.ID, &k.AdminID, &k.CredID, &k.PublicKey, &k.SignCount, &k.Name, &k.CreatedAt, &k.LastUsed); err != nil {
return nil, err
}
out = append(out, k)
}
return out, rows.Err()
}
// PasswordOff tells an admin who turned the password off: then only a
// passkey signs them in. It holds only while they have one.
func (db *DB) PasswordOff(adminID int64) bool {
var off int
_ = db.sql.QueryRow(`SELECT password_off FROM admins WHERE id = ?`, adminID).Scan(&off)
return off == 1 && db.CountAdminKeys(adminID) > 0
}
// AdminPin is the hash of the admin's PIN, "" when there is none.
func (db *DB) AdminPin(adminID int64) string {
var h string
_ = db.sql.QueryRow(`SELECT pin_hash FROM admins WHERE id = ?`, adminID).Scan(&h)
return h
}
func (db *DB) SetAdminPin(adminID int64, pinHash string) error {
_, err := db.sql.Exec(`UPDATE admins SET pin_hash = ? WHERE id = ?`, pinHash, adminID)
return err
}
func (db *DB) SetAdminPassword(adminID int64, passHash string) error {
_, err := db.sql.Exec(`UPDATE admins SET pass_hash = ? WHERE id = ?`, passHash, adminID)
return err
}
func (db *DB) SetPasswordOff(adminID int64, off bool) error {
v := 0
if off {
v = 1
}
_, err := db.sql.Exec(`UPDATE admins SET password_off = ? WHERE id = ?`, v, adminID)
return err
}
// CountAdminKeys is how many passkeys an admin has.
func (db *DB) CountAdminKeys(adminID int64) int {
var n int
_ = db.sql.QueryRow(`SELECT COUNT(*) FROM admin_keys WHERE admin_id = ?`, adminID).Scan(&n)
return n
}
func (db *DB) AdminKeyByCredID(credID []byte) (*AdminKey, error) {
k := &AdminKey{}
err := db.sql.QueryRow(
`SELECT id, admin_id, cred_id, public_key, sign_count, name, created_at, last_used
FROM admin_keys WHERE cred_id = ?`, credID).
Scan(&k.ID, &k.AdminID, &k.CredID, &k.PublicKey, &k.SignCount, &k.Name, &k.CreatedAt, &k.LastUsed)
if err == sql.ErrNoRows {
return nil, nil
}
return k, err
}
func (db *DB) UseAdminKey(id int64, signCount uint32) error {
_, err := db.sql.Exec(`UPDATE admin_keys SET sign_count = ?, last_used = ? WHERE id = ?`, signCount, time.Now().Unix(), id)
return err
}
// DeleteAdminKey removes a passkey; with the last one gone, the password
// signs the admin in again.
func (db *DB) DeleteAdminKey(adminID, id int64) error {
if _, err := db.sql.Exec(`DELETE FROM admin_keys WHERE admin_id = ? AND id = ?`, adminID, id); err != nil {
return err
}
if db.CountAdminKeys(adminID) == 0 {
return db.SetPasswordOff(adminID, false)
}
return nil
}
func (db *DB) DeleteAdminKeys(adminID int64) error {
if _, err := db.sql.Exec(`DELETE FROM admin_keys WHERE admin_id = ?`, adminID); err != nil {
return err
}
return db.SetPasswordOff(adminID, false)
}
// --- admin sessions ---
func (db *DB) CreateSession(id string, adminID int64, ttl time.Duration) error {
@@ -198,12 +457,10 @@ func (db *DB) CreateSession(id string, adminID int64, ttl time.Duration) error {
// SessionAdmin returns the admin for a live (non-expired) session, or nil.
func (db *DB) SessionAdmin(sessionID string) (*Admin, error) {
a := &Admin{}
err := db.sql.QueryRow(
`SELECT a.id, a.username, a.pass_hash
a, err := scanAdmin(db.sql.QueryRow(
`SELECT a.id, a.username, a.pass_hash, a.role, a.pw_locked
FROM admin_sessions s JOIN admins a ON a.id = s.admin_id
WHERE s.id = ? AND s.expires_at > ?`, sessionID, time.Now().Unix()).
Scan(&a.ID, &a.Username, &a.PassHash)
WHERE s.id = ? AND s.expires_at > ?`, sessionID, time.Now().Unix()))
if err == sql.ErrNoRows {
return nil, nil
}
@@ -228,6 +485,7 @@ type Agent struct {
Version string
RunUser string // OS user the agent process runs as (reported by the agent)
LoginUser string // OS user terminals are opened as; "" = RunUser
Transport string // auto, ws or lp: how the agent is to reach us
LastSeen time.Time
}
@@ -238,8 +496,9 @@ func (a Agent) TerminalUser() string {
return firstNonEmpty(a.LoginUser, a.RunUser, "root")
}
// Online reports whether the agent polled recently enough to be considered up.
func (a Agent) Online() bool {
// recentlySeen is the fallback for "online" right after a server start, while
// the hub has not seen the agents' polls yet (see Server.online).
func (a Agent) recentlySeen() bool {
return time.Since(a.LastSeen) < 45*time.Second
}
@@ -256,8 +515,8 @@ func (db *DB) GetAgent(id string) (*Agent, error) {
a := &Agent{}
var last int64
err := db.sql.QueryRow(
`SELECT id, name, hostname, os, arch, token_hash, version, run_user, login_user, last_seen FROM agents WHERE id = ?`, id).
Scan(&a.ID, &a.Name, &a.Hostname, &a.OS, &a.Arch, &a.TokenHash, &a.Version, &a.RunUser, &a.LoginUser, &last)
`SELECT id, name, hostname, os, arch, token_hash, version, run_user, login_user, transport, last_seen FROM agents WHERE id = ?`, id).
Scan(&a.ID, &a.Name, &a.Hostname, &a.OS, &a.Arch, &a.TokenHash, &a.Version, &a.RunUser, &a.LoginUser, &a.Transport, &last)
if err == sql.ErrNoRows {
return nil, nil
}
@@ -288,6 +547,11 @@ func (db *DB) ListAgents() ([]Agent, error) {
return out, rows.Err()
}
func (db *DB) SetAgentTransport(id, transport string) error {
_, err := db.sql.Exec(`UPDATE agents SET transport = ? WHERE id = ?`, transport, id)
return err
}
func (db *DB) TouchAgent(id string) error {
_, err := db.sql.Exec(`UPDATE agents SET last_seen = ? WHERE id = ?`, time.Now().Unix(), id)
return err
@@ -308,6 +572,9 @@ func (db *DB) SetAgentLoginUser(id, user string) error {
func (db *DB) DeleteAgent(id string) error {
_, err := db.sql.Exec(`DELETE FROM agents WHERE id = ?`, id)
if err == nil {
_, _ = db.sql.Exec(`DELETE FROM admin_agents WHERE agent_id = ?`, id)
}
return err
}
@@ -327,25 +594,23 @@ type ForwardRule struct {
HostPort int
ListenAddr string
Proto string
NoUfw bool // the panel opens no ufw rule for this forward
Label string // optional note shown next to the rule
}
func (db *DB) CreateForwardRule(r ForwardRule) (int64, error) {
if r.Proto == "" {
r.Proto = "http"
}
res, err := db.sql.Exec(
`INSERT INTO forward_rules(agent_id, client_port, host_port, listen_addr, proto, created_at)
VALUES(?, ?, ?, ?, ?, ?)`,
r.AgentID, r.ClientPort, r.HostPort, r.ListenAddr, r.Proto, time.Now().Unix())
if err != nil {
return 0, err
}
return res.LastInsertId()
return db.sql.insert(
`INSERT INTO forward_rules(agent_id, client_port, host_port, listen_addr, proto, no_ufw, label, created_at)
VALUES(?, ?, ?, ?, ?, ?, ?, ?)`,
r.AgentID, r.ClientPort, r.HostPort, r.ListenAddr, r.Proto, boolInt(r.NoUfw), r.Label, time.Now().Unix())
}
func (db *DB) ListForwardRules() ([]ForwardRule, error) {
rows, err := db.sql.Query(
`SELECT id, agent_id, client_port, host_port, listen_addr, proto FROM forward_rules ORDER BY id`)
`SELECT id, agent_id, client_port, host_port, listen_addr, proto, no_ufw, label FROM forward_rules ORDER BY id`)
if err != nil {
return nil, err
}
@@ -353,7 +618,7 @@ func (db *DB) ListForwardRules() ([]ForwardRule, error) {
var out []ForwardRule
for rows.Next() {
var r ForwardRule
if err := rows.Scan(&r.ID, &r.AgentID, &r.ClientPort, &r.HostPort, &r.ListenAddr, &r.Proto); err != nil {
if err := rows.Scan(&r.ID, &r.AgentID, &r.ClientPort, &r.HostPort, &r.ListenAddr, &r.Proto, &r.NoUfw, &r.Label); err != nil {
return nil, err
}
out = append(out, r)
@@ -378,8 +643,8 @@ func (db *DB) ListForwardRulesByAgent(agentID string) ([]ForwardRule, error) {
func (db *DB) GetForwardRule(id int64) (*ForwardRule, error) {
r := &ForwardRule{}
err := db.sql.QueryRow(
`SELECT id, agent_id, client_port, host_port, listen_addr, proto FROM forward_rules WHERE id = ?`, id).
Scan(&r.ID, &r.AgentID, &r.ClientPort, &r.HostPort, &r.ListenAddr, &r.Proto)
`SELECT id, agent_id, client_port, host_port, listen_addr, proto, no_ufw, label FROM forward_rules WHERE id = ?`, id).
Scan(&r.ID, &r.AgentID, &r.ClientPort, &r.HostPort, &r.ListenAddr, &r.Proto, &r.NoUfw, &r.Label)
if err == sql.ErrNoRows {
return nil, nil
}
@@ -397,7 +662,20 @@ type UfwRule struct {
ID int64
Port int
Proto string
RuleID int64 // bound forward rule id, or 0 for shared/manual (e.g. tunnel 8446)
RuleID int64 // bound forward rule id, 0 for shared (the tunnel port), ufwManual for one added in Settings
}
// ufwManual is the RuleID of a ufw rule the admin added by hand: no forward
// owns it, so only the admin removes it.
const ufwManual = -1
// Manual reports whether the admin added the rule by hand.
func (r UfwRule) Manual() bool { return r.RuleID == ufwManual }
// SetUfwRuleManual hands a tracked rule over to the admin.
func (db *DB) SetUfwRuleManual(port int, proto string) error {
_, err := db.sql.Exec(`UPDATE ufw_rules SET rule_id = ? WHERE port = ? AND proto = ?`, ufwManual, port, proto)
return err
}
func (db *DB) AddUfwRule(port int, proto string, ruleID int64) error {
@@ -469,3 +747,22 @@ func (db *DB) Audit(adminID int64, agentID, action string) {
`INSERT INTO audit(admin_id, agent_id, action, at) VALUES(?, ?, ?, ?)`,
adminID, agentID, action, time.Now().Unix())
}
func boolInt(b bool) int {
if b {
return 1
}
return 0
}
// wipeDeleted makes what was deleted leave the database's files: with
// secure_delete the freed pages are overwritten, and the write-ahead log,
// which still holds the rows as they were written, is emptied (SQLite only;
// PostgreSQL cleans up for itself). The attachments of a chat deleted are
// among that.
func (db *DB) wipeDeleted() {
if db.sql.pg {
return
}
_, _ = db.sql.Exec(`PRAGMA wal_checkpoint(TRUNCATE)`)
}
+251
View File
@@ -0,0 +1,251 @@
package server
import (
"context"
"database/sql"
"encoding/json"
"errors"
"fmt"
"log"
"net/http"
"net/url"
"os"
"strconv"
"strings"
"time"
_ "github.com/jackc/pgx/v5/stdlib"
)
// The panel keeps its data in SQLite by default, or in PostgreSQL when the
// admins choose so in Настройки. The choice lives in a small file beside the
// SQLite database (<db>.backend.json, readable by the server only) and
// takes effect on the next start; the first start on an empty PostgreSQL
// copies everything over from the SQLite file, which stays as it was.
type PGConf struct {
Host string `json:"host"`
Port int `json:"port"`
Name string `json:"name"`
User string `json:"user"`
Password string `json:"password"`
SSLMode string `json:"sslmode"`
}
type backendConf struct {
Type string `json:"type"` // sqlite or postgres
PG *PGConf `json:"pg,omitempty"`
}
func (c PGConf) DSN() string {
port := c.Port
if port == 0 {
port = 5432
}
ssl := c.SSLMode
if ssl == "" {
ssl = "prefer"
}
u := url.URL{Scheme: "postgres", User: url.UserPassword(c.User, c.Password), Host: c.Host + ":" + strconv.Itoa(port), Path: "/" + c.Name}
u.RawQuery = url.Values{"sslmode": {ssl}, "connect_timeout": {"10"}}.Encode()
return u.String()
}
func backendPath(dbPath string) string { return dbPath + ".backend.json" }
func readBackend(dbPath string) backendConf {
var c backendConf
if b, err := os.ReadFile(backendPath(dbPath)); err == nil {
_ = json.Unmarshal(b, &c)
}
if c.Type != "postgres" || c.PG == nil {
c = backendConf{Type: "sqlite"}
}
return c
}
func writeBackend(dbPath string, c backendConf) error {
b, _ := json.MarshalIndent(c, "", " ")
return os.WriteFile(backendPath(dbPath), b, 0o600)
}
// OpenStore opens the database the admins chose: SQLite at path, or the
// PostgreSQL named beside it.
func OpenStore(path string) (*DB, error) {
c := readBackend(path)
if c.Type == "postgres" {
return OpenPostgres(*c.PG, path)
}
return OpenDB(path)
}
// pingPG checks that PostgreSQL c answers.
func pingPG(c PGConf) error {
d, err := sql.Open("pgx", c.DSN())
if err != nil {
return err
}
defer d.Close()
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
return d.PingContext(ctx)
}
// OpenPostgres opens PostgreSQL c, brings its tables up to date and, when it
// has no users yet, copies over the SQLite database at sqlitePath.
func OpenPostgres(c PGConf, sqlitePath string) (*DB, error) {
d, err := sql.Open("pgx", c.DSN())
if err != nil {
return nil, err
}
d.SetMaxOpenConns(16)
d.SetMaxIdleConns(4)
d.SetConnMaxIdleTime(5 * time.Minute)
if err := d.Ping(); err != nil {
return nil, fmt.Errorf("PostgreSQL: %w", err)
}
db := &DB{sql: &dbx{DB: d, pg: true}, path: sqlitePath}
if err := db.migrate(); err != nil {
return nil, fmt.Errorf("PostgreSQL: %w", err)
}
var n int
_ = db.sql.QueryRow(`SELECT COUNT(*) FROM admins`).Scan(&n)
if _, err := os.Stat(sqlitePath); n == 0 && err == nil {
src, err := OpenDB(sqlitePath)
if err != nil {
return nil, err
}
defer src.Close()
if err := copyDB(src, db); err != nil {
return nil, fmt.Errorf("перенос из SQLite: %w", err)
}
log.Printf("db: data copied from %s to PostgreSQL", sqlitePath)
}
return db, nil
}
// copyTables are copied parents first, for the foreign keys.
var copyTables = []string{"config", "admins", "agents", "admin_sessions", "audit", "ufw_rules", "admin_keys",
"forward_rules", "admin_agents", "ssh_hosts", "ai_chats", "ai_messages"}
// copyDB copies every row of src into dst (an empty PostgreSQL), in one
// transaction, then moves the id counters past the copied ids.
func copyDB(src, dst *DB) error {
tx, err := dst.sql.DB.Begin()
if err != nil {
return err
}
defer tx.Rollback()
for _, t := range copyTables {
rows, err := src.sql.Query(`SELECT * FROM ` + t)
if err != nil {
return err
}
cols, _ := rows.Columns()
ph := make([]string, len(cols))
for i := range ph {
ph[i] = "$" + strconv.Itoa(i+1)
}
ins := `INSERT INTO ` + t + ` (` + strings.Join(cols, ", ") + `) VALUES (` + strings.Join(ph, ", ") + `)`
for rows.Next() {
vals := make([]any, len(cols))
ptrs := make([]any, len(cols))
for i := range vals {
ptrs[i] = &vals[i]
}
if err := rows.Scan(ptrs...); err != nil {
rows.Close()
return err
}
if _, err := tx.Exec(ins, vals...); err != nil {
rows.Close()
return fmt.Errorf("%s: %w", t, err)
}
}
rows.Close()
if err := rows.Err(); err != nil {
return err
}
}
if err := tx.Commit(); err != nil {
return err
}
// The id counters past the copied ids, where a table has one.
for _, t := range copyTables {
var seq sql.NullString
err := dst.sql.DB.QueryRow(`SELECT pg_get_serial_sequence($1, column_name) FROM information_schema.columns
WHERE table_name = $1 AND column_name = 'id' AND column_default LIKE 'nextval%'`, t).Scan(&seq)
if err != nil || !seq.Valid {
continue
}
if _, err := dst.sql.DB.Exec(`SELECT setval($1, COALESCE((SELECT MAX(id) FROM `+t+`), 0) + 1, false)`, seq.String); err != nil {
return err
}
}
return nil
}
// Backend tells which database is in use and which is chosen for the next
// start.
func (db *DB) Backend() (now string, chosen backendConf) {
now = "sqlite"
if db.sql.pg {
now = "postgres"
}
return now, readBackend(db.path)
}
var errNoPGHost = errors.New("укажите хост, базу и пользователя PostgreSQL")
// POST /settings/db - the database for the next start; PostgreSQL is tried
// first and kept only if it answers.
func (s *Server) handleSetDB(w http.ResponseWriter, r *http.Request) {
if !checkCSRF(r) {
http.Error(w, "bad csrf", http.StatusForbidden)
return
}
back := func(q string) { http.Redirect(w, r, "/settings?"+q+"#db", http.StatusSeeOther) }
if r.FormValue("db_type") != "postgres" {
if err := writeBackend(s.db.path, backendConf{Type: "sqlite"}); err != nil {
back("err=" + url.QueryEscape(err.Error()))
return
}
back("db=saved")
return
}
old := readBackend(s.db.path).PG
c := PGConf{Host: strings.TrimSpace(r.FormValue("pg_host")), Name: strings.TrimSpace(r.FormValue("pg_name")),
User: strings.TrimSpace(r.FormValue("pg_user")), Password: r.FormValue("pg_password"), SSLMode: r.FormValue("pg_sslmode")}
c.Port, _ = strconv.Atoi(r.FormValue("pg_port"))
if c.Password == "" && old != nil {
c.Password = old.Password
}
if c.Host == "" || c.Name == "" || c.User == "" {
back("err=" + url.QueryEscape(errNoPGHost.Error()))
return
}
if err := pingPG(c); err != nil {
back("err=" + url.QueryEscape("PostgreSQL не отвечает: "+err.Error()))
return
}
if err := writeBackend(s.db.path, backendConf{Type: "postgres", PG: &c}); err != nil {
back("err=" + url.QueryEscape(err.Error()))
return
}
back("db=saved")
}
// POST /settings/db/restart - the server ends; its service (systemd,
// Restart=always) starts it again on the database chosen.
func (s *Server) handleRestart(w http.ResponseWriter, r *http.Request) {
if !checkCSRF(r) {
http.Error(w, "bad csrf", http.StatusForbidden)
return
}
log.Printf("restart asked from the panel")
http.Redirect(w, r, "/settings?restarted=1#db", http.StatusSeeOther)
go func() {
time.Sleep(700 * time.Millisecond)
os.Exit(0)
}()
}
+130
View File
@@ -0,0 +1,130 @@
package server
import (
"database/sql"
"encoding/json"
"os"
"testing"
"github.com/x-grk/longpolling-remote-access/internal/proto"
)
// On a real PostgreSQL (LPRA_TEST_PG: {"host","port","name","user","password",
// "sslmode"} of an empty database): the first start copies the SQLite data
// over, and the panel's queries work there.
func TestPostgres(t *testing.T) {
raw := os.Getenv("LPRA_TEST_PG")
if raw == "" {
t.Skip("LPRA_TEST_PG not set")
}
var c PGConf
if err := json.Unmarshal([]byte(raw), &c); err != nil {
t.Fatal(err)
}
// SQLite with a user, a machine, a rule, a saved host and some talk.
s, _ := newTestServer(t)
a, _ := s.db.AdminByUsername("adm")
if _, err := s.db.CreateForwardRule(ForwardRule{AgentID: "x1", ClientPort: 80, HostPort: 18080, ListenAddr: "0.0.0.0", Proto: "tcp", NoUfw: true}); err != nil {
t.Fatal(err)
}
if _, err := s.saveSSHHost(a.ID, savedHost{Label: "web", Host: "h", Port: 22, User: "root", Auth: "password"}, sshSecret{Password: "pw"}); err != nil {
t.Fatal(err)
}
chat := s.aiNewChat(a.ID)
s.aiAppendTo(a.ID, chat, "user", "привет")
_ = s.db.SetUserAgents(a.ID, []string{"x1"})
pgReset(t, c)
pg, err := OpenPostgres(c, s.db.path)
if err != nil {
t.Fatal(err)
}
defer pg.Close()
// Clean up for the next run.
defer func() {
for i := len(copyTables) - 1; i >= 0; i-- {
_, _ = pg.sql.Exec(`DROP TABLE IF EXISTS ` + copyTables[i] + ` CASCADE`)
}
}()
p := &Server{db: pg}
if got, _ := pg.AdminByUsername("adm"); got == nil || got.ID != a.ID {
t.Fatalf("admin not copied: %+v", got)
}
if ag, _ := pg.GetAgent("x1"); ag == nil {
t.Fatal("agent not copied")
}
rules, _ := pg.ListForwardRules()
if len(rules) != 1 || !rules[0].NoUfw {
t.Fatalf("rules: %+v", rules)
}
tg := &proto.SSHTarget{Saved: p.listSSHHosts(a.ID)[0].ID}
if err := p.fillSaved(a.ID, tg); err != nil || tg.Password != "pw" {
t.Fatalf("saved host: %+v %v", tg, err)
}
if h := p.aiHistoryOf(a.ID, chat, 10); len(h) != 1 || h[0].Content != "привет" {
t.Fatalf("talk: %+v", h)
}
// New rows go on after the copied ids.
id, err := pg.CreateUser("bob", "x", RoleUser)
if err != nil || id <= a.ID {
t.Fatalf("new user id %d, %v", id, err)
}
if err := pg.SetConfig("k", "v1"); err != nil {
t.Fatal(err)
}
if err := pg.SetConfig("k", "v2"); err != nil {
t.Fatal(err)
}
if v, _ := pg.GetConfig("k"); v != "v2" {
t.Fatalf("config upsert: %q", v)
}
if err := pg.SetUserAgents(id, []string{"x1", "x1"}); err != nil {
t.Fatalf("grants: %v", err)
}
p.aiAppendTo(a.ID, chat, "assistant", "ответ")
if h := p.aiHistoryOf(a.ID, chat, 10); len(h) != 2 {
t.Fatalf("talk after: %+v", h)
}
}
// Chats on PostgreSQL: past the limit the oldest go.
func TestPostgresChats(t *testing.T) {
raw := os.Getenv("LPRA_TEST_PG")
if raw == "" {
t.Skip("LPRA_TEST_PG not set")
}
var c PGConf
_ = json.Unmarshal([]byte(raw), &c)
s, _ := newTestServer(t)
pgReset(t, c)
pg, err := OpenPostgres(c, s.db.path)
if err != nil {
t.Fatal(err)
}
defer pg.Close()
defer func() {
for i := len(copyTables) - 1; i >= 0; i-- {
_, _ = pg.sql.Exec(`DROP TABLE IF EXISTS ` + copyTables[i] + ` CASCADE`)
}
}()
p := &Server{db: pg}
a, _ := pg.AdminByUsername("adm")
for i := 0; i < aiChatsKeep+3; i++ {
p.aiAppendTo(a.ID, p.aiNewChat(a.ID), "user", "q")
}
if n := len(p.aiChats(a.ID)); n != aiChatsKeep {
t.Fatalf("chats kept: %d", n)
}
}
// pgReset empties the test database: the copy runs only into an empty one.
func pgReset(t *testing.T, c PGConf) {
d, err := sql.Open("pgx", c.DSN())
if err != nil {
t.Fatal(err)
}
defer d.Close()
for i := len(copyTables) - 1; i >= 0; i-- {
_, _ = d.Exec(`DROP TABLE IF EXISTS ` + copyTables[i] + ` CASCADE`)
}
}
+93
View File
@@ -0,0 +1,93 @@
package server
import (
"database/sql"
"strconv"
"strings"
)
// dbx is the database the server talks to: SQLite, or PostgreSQL. The
// queries are written once, for SQLite; on PostgreSQL each is rewritten
// (pgSQL) on its way.
type dbx struct {
*sql.DB
pg bool
}
func (d *dbx) q(query string) string {
if d.pg {
return pgSQL(query)
}
return query
}
func (d *dbx) Exec(query string, args ...any) (sql.Result, error) {
return d.DB.Exec(d.q(query), args...)
}
func (d *dbx) Query(query string, args ...any) (*sql.Rows, error) {
return d.DB.Query(d.q(query), args...)
}
func (d *dbx) QueryRow(query string, args ...any) *sql.Row {
return d.DB.QueryRow(d.q(query), args...)
}
func (d *dbx) Begin() (*txx, error) {
tx, err := d.DB.Begin()
return &txx{Tx: tx, pg: d.pg}, err
}
// insert runs an INSERT and gives the new row's id (LastInsertId on SQLite,
// RETURNING id on PostgreSQL, which has no LastInsertId).
func (d *dbx) insert(query string, args ...any) (int64, error) {
if d.pg {
var id int64
err := d.DB.QueryRow(pgSQL(query)+" RETURNING id", args...).Scan(&id)
return id, err
}
res, err := d.DB.Exec(query, args...)
if err != nil {
return 0, err
}
return res.LastInsertId()
}
type txx struct {
*sql.Tx
pg bool
}
func (t *txx) Exec(query string, args ...any) (sql.Result, error) {
if t.pg {
query = pgSQL(query)
}
return t.Tx.Exec(query, args...)
}
// pgSQL rewrites a query for SQLite into PostgreSQL's: $n placeholders,
// INSERT OR IGNORE as ON CONFLICT DO NOTHING, and in table definitions
// BIGSERIAL keys, BIGINT numbers (times are Unix seconds), BYTEA blobs and
// ADD COLUMN IF NOT EXISTS.
func pgSQL(q string) string {
if strings.HasPrefix(q, "INSERT OR IGNORE ") {
q = "INSERT " + strings.TrimPrefix(q, "INSERT OR IGNORE ") + " ON CONFLICT DO NOTHING"
}
if strings.Contains(q, "CREATE TABLE") || strings.Contains(q, "ALTER TABLE") {
q = strings.ReplaceAll(q, "INTEGER PRIMARY KEY AUTOINCREMENT", "BIGSERIAL PRIMARY KEY")
q = strings.ReplaceAll(q, "INTEGER", "BIGINT")
q = strings.ReplaceAll(q, "BLOB", "BYTEA")
q = strings.ReplaceAll(q, "ADD COLUMN ", "ADD COLUMN IF NOT EXISTS ")
}
var b strings.Builder
n, quoted := 0, false
for _, r := range q {
switch {
case r == '\'':
quoted = !quoted
case r == '?' && !quoted:
n++
b.WriteString("$" + strconv.Itoa(n))
continue
}
b.WriteRune(r)
}
return b.String()
}
+19
View File
@@ -0,0 +1,19 @@
package server
import "testing"
// Queries written for SQLite, as PostgreSQL takes them.
func TestPGRebind(t *testing.T) {
for _, c := range [][2]string{
{`SELECT a FROM t WHERE x = ? AND y = ?`, `SELECT a FROM t WHERE x = $1 AND y = $2`},
{`INSERT OR IGNORE INTO admin_agents(admin_id, agent_id) VALUES(?, ?)`, `INSERT INTO admin_agents(admin_id, agent_id) VALUES($1, $2) ON CONFLICT DO NOTHING`},
{`SELECT '?' , ?`, `SELECT '?' , $1`},
{"CREATE TABLE IF NOT EXISTS x (\n id INTEGER PRIMARY KEY AUTOINCREMENT,\n n INTEGER NOT NULL,\n b BLOB NOT NULL)",
"CREATE TABLE IF NOT EXISTS x (\n id BIGSERIAL PRIMARY KEY,\n n BIGINT NOT NULL,\n b BYTEA NOT NULL)"},
{`ALTER TABLE agents ADD COLUMN run_user TEXT NOT NULL DEFAULT ''`, `ALTER TABLE agents ADD COLUMN IF NOT EXISTS run_user TEXT NOT NULL DEFAULT ''`},
} {
if got := pgSQL(c[0]); got != c[1] {
t.Errorf("\n got %s\nwant %s", got, c[1])
}
}
}
+199
View File
@@ -0,0 +1,199 @@
package server
import (
"context"
"encoding/json"
"errors"
"html"
"io"
"mime"
"net"
"net/http"
"net/url"
"regexp"
"strconv"
"strings"
"time"
"unicode/utf8"
)
// fetch: one page by its address, for the assistant with «Поиск» on - to
// read what a search found. The page comes back as the model wants it: its
// title and main text, headings and lists kept, scripts, styles and menus
// gone, cut to fetchMaxChars. Only http(s), and never the server's own
// network: loopback, private and link-local addresses are refused at the
// moment of connecting, redirects included.
const fetchMaxChars = 12000
// fetchAllowPrivate lets tests reach their own server.
var fetchAllowPrivate = false
type fetchedPage struct {
URL string `json:"url"`
Title string `json:"title"`
Type string `json:"type"`
Text string `json:"text"`
Truncated bool `json:"truncated"`
}
func privateIP(ip net.IP) bool {
return ip.IsLoopback() || ip.IsPrivate() || ip.IsLinkLocalUnicast() || ip.IsLinkLocalMulticast() ||
ip.IsUnspecified() || ip.IsMulticast() || ip.IsInterfaceLocalMulticast()
}
var fetchClient = &http.Client{
Timeout: 20 * time.Second,
Transport: &http.Transport{
Proxy: nil,
DialContext: func(ctx context.Context, network, addr string) (net.Conn, error) {
host, port, err := net.SplitHostPort(addr)
if err != nil {
return nil, err
}
ips, err := net.DefaultResolver.LookupIP(ctx, "ip", host)
if err != nil {
return nil, err
}
for _, ip := range ips {
if privateIP(ip) && !fetchAllowPrivate {
continue
}
var d net.Dialer
return d.DialContext(ctx, network, net.JoinHostPort(ip.String(), port))
}
return nil, errors.New("адрес во внутренней сети - не открываю")
},
TLSHandshakeTimeout: 10 * time.Second,
ResponseHeaderTimeout: 15 * time.Second,
},
CheckRedirect: func(req *http.Request, via []*http.Request) error {
if len(via) >= 5 {
return errors.New("слишком много переадресаций")
}
if req.URL.Scheme != "http" && req.URL.Scheme != "https" {
return errors.New("переадресация не на http(s)")
}
return nil
},
}
func fetchPage(ctx context.Context, raw string) (fetchedPage, error) {
u, err := url.Parse(strings.TrimSpace(raw))
if err != nil || (u.Scheme != "http" && u.Scheme != "https") || u.Host == "" {
return fetchedPage{}, errors.New("нужен адрес http(s)://…")
}
req, _ := http.NewRequestWithContext(ctx, http.MethodGet, u.String(), nil)
req.Header.Set("User-Agent", "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0 Safari/537.36")
req.Header.Set("Accept", "text/html,text/plain,application/json;q=0.9,*/*;q=0.5")
req.Header.Set("Accept-Language", "ru,en;q=0.8")
resp, err := fetchClient.Do(req)
if err != nil {
return fetchedPage{}, err
}
defer resp.Body.Close()
if resp.StatusCode/100 != 2 {
return fetchedPage{}, errors.New("страница ответила " + resp.Status)
}
ct, _, _ := mime.ParseMediaType(resp.Header.Get("Content-Type"))
p := fetchedPage{URL: resp.Request.URL.String(), Type: ct}
b, _ := io.ReadAll(io.LimitReader(resp.Body, 3<<20))
switch {
case ct == "text/html" || ct == "application/xhtml+xml" || (ct == "" && strings.Contains(strings.ToLower(string(b[:min(len(b), 512)])), "<html")):
p.Title, p.Text = pageText(string(b))
case strings.HasPrefix(ct, "text/") || strings.Contains(ct, "json") || strings.Contains(ct, "xml") || strings.Contains(ct, "javascript"):
p.Text = strings.TrimSpace(string(b))
default:
p.Text = "(двоичный файл " + ct + ", " + strconv.Itoa(len(b)) + " байт - текста нет)"
}
if !utf8.ValidString(p.Text) {
p.Text = strings.ToValidUTF8(p.Text, "")
}
if r := []rune(p.Text); len(r) > fetchMaxChars {
p.Text, p.Truncated = string(r[:fetchMaxChars]), true
}
return p, nil
}
var (
reTitle = regexp.MustCompile(`(?is)<title[^>]*>(.*?)</title>`)
reDrop = regexp.MustCompile(`(?is)<(script|style|noscript|svg|nav|footer|header|aside|form|iframe|template|button|select)\b[^>]*>.*?</(script|style|noscript|svg|nav|footer|header|aside|form|iframe|template|button|select)>`)
reComment = regexp.MustCompile(`(?s)<!--.*?-->`)
reMain = regexp.MustCompile(`(?is)<(main|article)\b[^>]*>(.*)</(main|article)>`)
reHead = regexp.MustCompile(`(?is)<h([1-6])\b[^>]*>`)
reLi = regexp.MustCompile(`(?is)<li\b[^>]*>`)
reBlock = regexp.MustCompile(`(?is)</?(p|div|section|br|tr|table|ul|ol|pre|blockquote|h[1-6]|dd|dt|figcaption)\b[^>]*>`)
reCell = regexp.MustCompile(`(?is)</t[dh]>`)
reSpaces = regexp.MustCompile(`[ \t\x{00a0}]+`)
reBlanks = regexp.MustCompile(`\n{3,}`)
rePre = regexp.MustCompile(`(?is)<pre\b[^>]*>.*?</pre>`)
rePreMark = regexp.MustCompile("\x00pre[0-9]+\x00")
reWS = regexp.MustCompile(`\s+`)
)
// pageText is a page's title and its text, in Markdown-ish lines.
func pageText(page string) (string, string) {
title := ""
if m := reTitle.FindStringSubmatch(page); m != nil {
title = strings.TrimSpace(html.UnescapeString(reTags.ReplaceAllString(m[1], "")))
}
s := reComment.ReplaceAllString(page, "")
s = reDrop.ReplaceAllString(s, " ")
// Line breaks of the source mean nothing outside <pre>: one space. The
// <pre> blocks keep theirs (held aside meanwhile).
var pres []string
s = rePre.ReplaceAllStringFunc(s, func(p string) string {
pres = append(pres, p)
return "\x00pre" + strconv.Itoa(len(pres)-1) + "\x00"
})
s = reWS.ReplaceAllString(s, " ")
s = rePreMark.ReplaceAllStringFunc(s, func(m string) string {
i, _ := strconv.Atoi(strings.Trim(m, "\x00pre"))
return pres[i]
})
// The page's main part, when it marks one.
if m := reMain.FindStringSubmatch(s); m != nil && len(m[2]) > 200 {
s = m[2]
}
s = reHead.ReplaceAllStringFunc(s, func(t string) string {
n, _ := strconv.Atoi(reHead.FindStringSubmatch(t)[1])
return "\n\n" + strings.Repeat("#", n) + " "
})
s = reLi.ReplaceAllString(s, "\n- ")
s = reCell.ReplaceAllString(s, " | ")
s = reBlock.ReplaceAllString(s, "\n")
s = html.UnescapeString(reTags.ReplaceAllString(s, ""))
var lines []string
for _, l := range strings.Split(s, "\n") {
l = strings.TrimSpace(reSpaces.ReplaceAllString(l, " "))
if l == "-" || l == "|" {
continue
}
lines = append(lines, l)
}
text := reBlanks.ReplaceAllString(strings.Join(lines, "\n"), "\n\n")
return title, strings.TrimSpace(text)
}
// POST /api/panel/ai/fetch - {url}: the page as text, for a user with the
// assistant on.
func (s *Server) handleAIFetch(w http.ResponseWriter, r *http.Request) {
a := s.currentAdmin(r)
if !s.aiOn(a.ID) {
writeJSON(w, map[string]any{"ok": false, "error": "ассистент выключен в профиле"})
return
}
var in struct {
URL string `json:"url"`
}
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 8<<10)).Decode(&in); err != nil {
writeJSON(w, map[string]any{"ok": false, "error": "плохой запрос"})
return
}
p, err := fetchPage(r.Context(), in.URL)
if err != nil {
writeJSON(w, map[string]any{"ok": false, "error": err.Error()})
return
}
writeJSON(w, map[string]any{"ok": true, "data": p})
}
+78
View File
@@ -0,0 +1,78 @@
package server
import (
"context"
"fmt"
"net/http"
"net/http/httptest"
"strings"
"testing"
)
// A page as the model wants it: title, the main text with headings and
// lists, no scripts, styles or menus.
func TestPageText(t *testing.T) {
page := `<html><head><title>Go 1.24 &amp; more</title><style>p{color:red}</style>
<meta name="description" content="Release notes"></head><body>
<nav><a href="/">Home</a> <a href="/doc">Docs</a></nav>
<main><h1>Go 1.24</h1><p>Generic <b>type aliases</b> are here.</p><script>track()</script>
<ul><li>Swiss tables</li><li>weak pointers</li></ul></main><footer>© Google</footer></body></html>`
title, text := pageText(page)
if title != "Go 1.24 & more" {
t.Fatalf("title %q", title)
}
for _, want := range []string{"# Go 1.24", "Generic type aliases are here.", "- Swiss tables", "- weak pointers"} {
if !strings.Contains(text, want) {
t.Fatalf("no %q in:\n%s", want, text)
}
}
for _, not := range []string{"track()", "color:red", "Home", "© Google"} {
if strings.Contains(text, not) {
t.Fatalf("%q left in:\n%s", not, text)
}
}
}
// The server's own network is out of reach: no loopback, private or local
// addresses, whatever the name says.
func TestFetchRefusesPrivate(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { fmt.Fprint(w, "secret") }))
defer srv.Close()
for _, u := range []string{srv.URL, "http://localhost:1/", "http://10.0.0.1/", "http://169.254.169.254/latest/meta-data/", "file:///etc/passwd", "ftp://x/"} {
if _, err := fetchPage(context.Background(), u); err == nil {
t.Errorf("%s: fetched", u)
}
}
}
func TestFetchPage(t *testing.T) {
old := fetchAllowPrivate
fetchAllowPrivate = true
defer func() { fetchAllowPrivate = old }()
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/json" {
w.Header().Set("Content-Type", "application/json")
fmt.Fprint(w, `{"a":1}`)
return
}
w.Header().Set("Content-Type", "text/html; charset=utf-8")
fmt.Fprint(w, "<title>T</title><p>"+strings.Repeat("слово ", 5000)+"</p>")
}))
defer srv.Close()
p, err := fetchPage(context.Background(), srv.URL+"/page")
if err != nil || p.Title != "T" || !p.Truncated || len([]rune(p.Text)) > fetchMaxChars+10 {
t.Fatalf("page: %v %q %v %d", err, p.Title, p.Truncated, len(p.Text))
}
p, err = fetchPage(context.Background(), srv.URL+"/json")
if err != nil || p.Text != `{"a":1}` {
t.Fatalf("json: %+v %v", p, err)
}
}
// Lines broken in the page's source are one paragraph; code keeps its lines.
func TestPageTextJoinsWrapped(t *testing.T) {
_, text := pageText("<p>The latest Go release,\n version 1.24,\narrives in February.</p><pre>a\nb</pre>")
if !strings.Contains(text, "The latest Go release, version 1.24, arrives in February.") || !strings.Contains(text, "a\nb") {
t.Fatalf("%q", text)
}
}
+36 -15
View File
@@ -12,10 +12,11 @@ import (
// forwardManager owns the host-side listeners for port-forward rules. It
// supports three transports:
// - "http": tunneled over the HTTP long-poll channel (works anywhere nginx
// reaches, highest latency);
// - "tcp": direct TCP multiplexed over the 8446 tunnel (low latency);
// - "udp": direct UDP over the 8446 channel.
// - "http": over the agent's own link to the panel - its WebSocket, or the
// HTTP long polls while there is none (works anywhere nginx reaches,
// highest latency);
// - "tcp": direct TCP multiplexed over the tunnel port (low latency);
// - "udp": direct UDP over the tunnel port.
type forwardManager struct {
hub *Hub
tun *tunnelHub
@@ -92,18 +93,19 @@ func (m *forwardManager) acceptTCP(rule ForwardRule, ln net.Listener) {
}
}
// bridgeDirect wires a connection to the agent's low-latency 8446 tunnel.
// bridgeDirect wires a connection to the agent's low-latency direct tunnel.
func (m *forwardManager) bridgeDirect(rule ForwardRule, conn net.Conn) {
at := m.tun.get(rule.AgentID)
if at == nil {
// No direct tunnel (agent offline or 8446 unreachable).
// No direct tunnel (agent offline or the tunnel port unreachable).
conn.Close()
return
}
at.openStream(rule.ClientPort, conn)
}
// bridgeHTTP wires a connection through the HTTP long-poll hub session.
// bridgeHTTP wires a connection through a hub session: the agent carries it
// over its WebSocket, or by long polls while it has none.
func (m *forwardManager) bridgeHTTP(rule ForwardRule, conn net.Conn) {
sess := m.hub.CreateForward(rule.AgentID, rule.ClientPort)
defer m.hub.CloseSession(sess.ID)
@@ -127,20 +129,39 @@ func (m *forwardManager) bridgeHTTP(rule ForwardRule, conn net.Conn) {
}
}()
for {
select {
case <-sess.Done():
return
case out, ok := <-sess.Out():
if !ok || out.Exit {
return
// write passes one message on; false: the stream is over.
write := func(out proto.OutMessage) bool {
if out.Exit {
return false
}
if out.Data != "" {
b, err := base64.StdEncoding.DecodeString(out.Data)
if err != nil {
return
return false
}
if _, err := conn.Write(b); err != nil {
return false
}
}
return true
}
for {
select {
case out, ok := <-sess.Out():
if !ok || !write(out) {
return
}
case <-sess.Done():
// The agent's end of stream closes the session at once, while
// what it sent before may still be queued (a download read
// faster than the browser takes it): that goes out first.
for {
select {
case out := <-sess.Out():
if !write(out) {
return
}
default:
return
}
}
+49
View File
@@ -0,0 +1,49 @@
package server
import (
"bytes"
"encoding/base64"
"io"
"net"
"testing"
"time"
"github.com/x-grk/longpolling-remote-access/internal/proto"
)
// An http-lp download the agent has finished (its last chunk, then the end)
// reaches the browser whole: the session closing does not cut off what is
// still queued for it.
func TestForwardHTTPDrainsAfterClose(t *testing.T) {
hub := NewHub()
m := &forwardManager{hub: hub}
user, host := net.Pipe()
go m.bridgeHTTP(ForwardRule{AgentID: "a1", ClientPort: 80, Proto: "http"}, host)
var sess *Session
for i := 0; i < 100 && sess == nil; i++ {
hub.mu.Lock()
for _, s := range hub.sessions {
sess = s
}
hub.mu.Unlock()
time.Sleep(10 * time.Millisecond)
}
if sess == nil {
t.Fatal("no session")
}
want := bytes.Repeat([]byte("0123456789abcdef"), 4096) // 64 KiB in 64 chunks
go func() {
for i := 0; i < len(want); i += 1024 {
sess.PushOut(proto.OutMessage{Data: base64.StdEncoding.EncodeToString(want[i : i+1024])})
}
sess.PushOut(proto.OutMessage{Exit: true})
sess.close() // as the server does on the agent's end of stream
}()
time.Sleep(50 * time.Millisecond) // all of it queued, the session closed, before the browser reads
_ = user.SetReadDeadline(time.Now().Add(5 * time.Second))
got, _ := io.ReadAll(user)
if !bytes.Equal(got, want) {
t.Fatalf("got %d of %d bytes", len(got), len(want))
}
}
+94
View File
@@ -0,0 +1,94 @@
package server
import (
"bytes"
"html/template"
"log"
"net/http"
"github.com/x-grk/longpolling-remote-access/internal/proto"
)
// A panel page is either a fragment - its content alone, for the shell to put
// in place without reloading - or the shell with that content already in it.
func isFrag(r *http.Request) bool { return r.URL.Query().Get("frag") == "1" }
// pageView is the shell's view a page is shown in; the machine pages (Порты,
// Статус, Изменить) open as a dialog over the machines view.
var pageView = map[string]string{"dashboard": "dashboard", "settings": "settings"}
func (s *Server) renderPage(w http.ResponseWriter, r *http.Request, name string, data map[string]any) {
w.Header().Set("Content-Type", "text/html; charset=utf-8")
w.Header().Set("Cache-Control", "no-store")
if data == nil {
data = map[string]any{}
}
data["IsAdmin"] = s.currentAdmin(r).IsAdmin()
if !isFrag(r) {
if _, ok := pageView[name]; !ok {
// A machine page asked for whole: the machines view with it in
// a dialog.
s.shell(w, r, map[string]any{"View": "dashboard", "Dialog": r.URL.Path})
return
}
}
var body bytes.Buffer
if err := s.tmpl.ExecuteTemplate(&body, name+"-body", data); err != nil {
log.Printf("template %s: %v", name, err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
if isFrag(r) {
_, _ = w.Write(body.Bytes())
return
}
data["View"] = pageView[name]
data["Body"] = template.HTML(body.String())
s.shell(w, r, data)
}
// shell writes the shell page with data (View, Body, Open, Dialog).
func (s *Server) shell(w http.ResponseWriter, r *http.Request, data map[string]any) {
data["IsAdmin"] = s.currentAdmin(r).IsAdmin()
data["EditLimit"] = proto.FSReadCap
if a := s.currentAdmin(r); a != nil {
data["Prefs"] = s.prefs(a.ID)
}
if data["View"] == "dashboard" && data["Body"] == nil {
// The machines list under a dialog: draw it too.
var body bytes.Buffer
if err := s.tmpl.ExecuteTemplate(&body, "dashboard-body", s.dashboardData(r)); err == nil {
data["Body"] = template.HTML(body.String())
}
}
w.Header().Set("Content-Type", "text/html; charset=utf-8")
w.Header().Set("Cache-Control", "no-store")
if err := s.tmpl.ExecuteTemplate(w, "app.html", data); err != nil {
log.Printf("template app.html: %v", err)
}
}
// handleShell serves the shell alone: the terminal view, or with a pane of
// a machine opened at once (/agents/{id}/terminal, /agents/{id}/files).
func (s *Server) handleShell(w http.ResponseWriter, r *http.Request) {
data := map[string]any{"View": "terminal"}
if id := r.PathValue("id"); id != "" {
a, _ := s.db.GetAgent(id)
if a == nil {
http.NotFound(w, r)
return
}
kind := "term"
switch r.PathValue("kind") {
case "", "terminal":
case "files":
kind = "files"
default:
http.NotFound(w, r)
return
}
data["Open"] = map[string]any{"kind": kind, "agent": map[string]string{"id": a.ID, "name": a.Name, "host": a.Hostname, "user": a.TerminalUser()}}
}
s.shell(w, r, data)
}
+159
View File
@@ -0,0 +1,159 @@
package server
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
)
// newTestServer is a server on a fresh database with an admin signed in (the
// cookie) and one machine, x1.
func newTestServer(t *testing.T) (*Server, *http.Cookie) {
t.Helper()
db, err := OpenDB(t.TempDir() + "/t.db")
if err != nil {
t.Fatal(err)
}
s, err := New(Options{DB: db, DistDir: t.TempDir()})
if err != nil {
t.Fatal(err)
}
hash, _ := hashPassword("Passw0rd12345")
if err := db.CreateAdmin("adm", hash); err != nil {
t.Fatal(err)
}
_ = db.SetConfig(cfgSetupDone, "1")
a, _ := db.AdminByUsername("adm")
if err := db.CreateSession("sid-test", a.ID, sessionTTL); err != nil {
t.Fatal(err)
}
if err := db.CreateAgent(Agent{ID: "x1", Name: "t", Hostname: "h", OS: "linux", Arch: "amd64", TokenHash: "x"}); err != nil {
t.Fatal(err)
}
return s, &http.Cookie{Name: sessionCookie, Value: "sid-test"}
}
func get(t *testing.T, s *Server, c *http.Cookie, path string) *httptest.ResponseRecorder {
t.Helper()
req := httptest.NewRequest(http.MethodGet, path, nil)
req.AddCookie(c)
w := httptest.NewRecorder()
s.Handler().ServeHTTP(w, req)
return w
}
func TestFragmentHasNoChrome(t *testing.T) {
s, cookie := newTestServer(t)
w := get(t, s, cookie, "/settings?frag=1")
body := w.Body.String()
if w.Code != 200 || strings.Contains(body, "<html") || strings.Contains(body, "topbar") {
t.Fatalf("fragment: code %d, has chrome: %q", w.Code, body[:min(200, len(body))])
}
if !strings.Contains(body, `data-module="settings"`) {
t.Fatalf("fragment has no module root")
}
}
func TestShellRoutes(t *testing.T) {
s, cookie := newTestServer(t)
for _, p := range []string{"/", "/workspace", "/terminal", "/settings", "/agents/x1/files", "/agents/x1/terminal"} {
if w := get(t, s, cookie, p); !strings.Contains(w.Body.String(), `id="app"`) {
t.Errorf("%s: not the shell (code %d)", p, w.Code)
}
}
}
func TestSSHFSOpen(t *testing.T) {
s, admin := newTestServer(t)
// Long gone: the machine is offline.
if _, err := s.db.sql.Exec(`UPDATE agents SET last_seen = 0`); err != nil {
t.Fatal(err)
}
post := func(c *http.Cookie) *httptest.ResponseRecorder {
req := httptest.NewRequest(http.MethodPost, "/api/panel/agents/x1/sshfs/open", strings.NewReader(`{"ssh":{"host":"h","user":"u","password":"p"}}`))
req.AddCookie(c)
w := httptest.NewRecorder()
s.Handler().ServeHTTP(w, req)
return w
}
// The machine exists but is offline: the route answers in JSON.
if w := post(admin); w.Code != 200 || !strings.Contains(w.Body.String(), "офлайн") {
t.Fatalf("admin: %d %s", w.Code, w.Body.String())
}
// A user without the machine does not reach it.
hash, _ := hashPassword("userpass123")
id, _ := s.db.CreateUser("bob", hash, RoleUser)
_ = s.db.CreateSession("sid-bob", id, sessionTTL)
if w := post(&http.Cookie{Name: sessionCookie, Value: "sid-bob"}); w.Code != http.StatusNotFound {
t.Fatalf("a user without the machine got %d", w.Code)
}
}
// Renaming from the machine dialog keeps it open on «Изменить».
func TestRenameStaysOnEdit(t *testing.T) {
s, c := newTestServer(t)
tok := "tok-test"
var req *http.Request
req = httptest.NewRequest(http.MethodPost, "/agents/x1/rename", strings.NewReader("csrf="+tok+"&name=new"))
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
req.AddCookie(c)
req.AddCookie(&http.Cookie{Name: csrfCookie, Value: tok})
w := httptest.NewRecorder()
s.Handler().ServeHTTP(w, req)
if loc := w.Header().Get("Location"); w.Code != http.StatusSeeOther || loc != "/agents/x1/edit" {
t.Fatalf("rename: %d to %q", w.Code, loc)
}
}
// Настройки show the assistant's model as it is set, key included.
func TestSettingsShowAI(t *testing.T) {
s, c := newTestServer(t)
_ = s.db.SetConfig(cfgAIKey, "key-123")
body := get(t, s, c, "/settings?frag=1").Body.String()
if !strings.Contains(body, `value="key-123"`) || !strings.Contains(body, `value="`+DefaultAIURL+`"`) {
t.Fatal("the AI card is empty")
}
}
// The look of the panel is each user's: kept on the server, only known
// settings with known values, and given to the page at once.
func TestViewPrefs(t *testing.T) {
s, c := newTestServer(t)
post := func(body string) string {
req := httptest.NewRequest(http.MethodPost, "/api/panel/account/prefs", strings.NewReader(body))
req.AddCookie(c)
w := httptest.NewRecorder()
s.Handler().ServeHTTP(w, req)
return w.Body.String()
}
if got := post(`{"tabIcons":"right","termFont":16,"compact":true,"motion":"off","evil":"<x>","fileTime":"maybe"}`); !strings.Contains(got, `"ok":true`) {
t.Fatal(got)
}
got := get(t, s, c, "/api/panel/account/prefs").Body.String()
if !strings.Contains(got, `"tabIcons":"right"`) || !strings.Contains(got, `"termFont":16`) || strings.Contains(got, "evil") || strings.Contains(got, "maybe") {
t.Fatalf("prefs: %s", got)
}
if body := get(t, s, c, "/workspace").Body.String(); !strings.Contains(body, `"tabIcons":"right"`) {
t.Fatal("the page does not get the prefs")
}
if got := post(`{"termFont":99}`); !strings.Contains(got, `"ok":true`) || strings.Contains(get(t, s, c, "/api/panel/account/prefs").Body.String(), "99") {
t.Fatal("a font size out of range was kept")
}
post(`{"openHere":"off"}`)
if got := get(t, s, c, "/api/panel/account/prefs").Body.String(); !strings.Contains(got, `"openHere":"off"`) {
t.Fatalf("openHere off not kept: %s", got)
}
post(`{"aiModelPick":"always","aiModel":"a1b2c3"}`)
if got := get(t, s, c, "/api/panel/account/prefs").Body.String(); !strings.Contains(got, `"aiModelPick":"always"`) || !strings.Contains(got, `"aiModel":"a1b2c3"`) {
t.Fatalf("the model choice not kept: %s", got)
}
post(`{"aiModelPick":"often","aiModel":"<x>"}`)
if got := get(t, s, c, "/api/panel/account/prefs").Body.String(); strings.Contains(got, "aiModel") {
t.Fatalf("an unknown model choice was kept: %s", got)
}
post(`{"openHere":"sometimes"}`)
if got := get(t, s, c, "/api/panel/account/prefs").Body.String(); strings.Contains(got, "openHere") {
t.Fatalf("an unknown openHere was kept: %s", got)
}
}
+642
View File
@@ -0,0 +1,642 @@
package server
import (
"context"
"encoding/base64"
"encoding/json"
"errors"
"io"
"net/http"
"path"
"strconv"
"strings"
"sync"
"time"
"github.com/x-grk/longpolling-remote-access/internal/proto"
)
// Files go between the browser and the agent in pieces. Each piece is a
// request of its own, small enough for any proxy in front (Cloudflare takes
// 100 MB a request at most), and one that fails is sent again instead of the
// whole file. Several pieces are in flight at once, so a good link stays as
// busy as it would with one long stream. The server holds only a few pieces
// of a transfer: whoever sends waits while they wait to be taken.
//
// A download keeps one plain response towards the browser, which a proxy
// passes on whatever its size: links, previews and the hex editor read it.
const (
// xferChunk is the size of a piece.
xferChunk = 4 << 20
// xferBuffer is how many pieces of a transfer the server holds.
xferBuffer = 8
// xferWait is how long a request waits for its piece, or for room for it,
// before it is told to ask again: well under a proxy's timeout.
xferWait = 50 * time.Second
// xferIdle ends a transfer nobody has touched for this long.
xferIdle = 10 * time.Minute
// xferTimeout bounds one transfer, however slow the links.
xferTimeout = 2 * time.Hour
)
// fsLegacyCap is the most an agent that predates the pieces can take in or
// hand over: the file goes base64 inside a task then.
const fsLegacyCap = 32 << 20
// xfer is one file on its way.
type xfer struct {
id, agentID string
mu sync.Mutex
changed chan struct{} // closed and replaced on every change
size int64 // -1 until known
n int // pieces in all, once the size is known
chunks map[int][]byte
// low is the first piece still to go: the ones below it were written by
// the agent (upload) or to the browser (download).
low int
sized chan struct{} // closed once the size is known
err string // why the transfer failed
done bool
result proto.TaskResult
last time.Time
}
func pieces(size int64) int {
if size <= 0 {
return 1 // an empty file is one empty piece
}
return int((size + xferChunk - 1) / xferChunk)
}
// pieceLen is how long piece i of the file is.
func (x *xfer) pieceLen(i int) int64 {
return min(int64(xferChunk), x.size-int64(i)*xferChunk)
}
// notify wakes whoever waits on the transfer; x.mu is held.
func (x *xfer) notify() {
x.last = time.Now()
close(x.changed)
x.changed = make(chan struct{})
}
// await waits until ready() holds and returns true with x.mu held; false,
// with it free, when the wait ran out or ctx ended.
func (x *xfer) await(ctx context.Context, d time.Duration, ready func() bool) bool {
t := time.NewTimer(d)
defer t.Stop()
for {
x.mu.Lock()
if ready() {
return true
}
ch := x.changed
x.mu.Unlock()
select {
case <-ch:
case <-t.C:
return false
case <-ctx.Done():
return false
}
}
}
func (x *xfer) fail(msg string) {
x.mu.Lock()
if x.err == "" && !x.done {
x.err = msg
x.chunks = nil
x.notify()
}
x.mu.Unlock()
}
func (x *xfer) finish(res proto.TaskResult) {
x.mu.Lock()
x.done, x.result, x.chunks = true, res, nil
x.notify()
x.mu.Unlock()
}
// setSize records the file's size; false when it differs from the one known.
func (x *xfer) setSize(size int64) bool {
x.mu.Lock()
defer x.mu.Unlock()
if x.size >= 0 {
return x.size == size
}
x.size, x.n = size, pieces(size)
close(x.sized)
x.notify()
return true
}
// put stores piece i once there is room for it, and answers with the HTTP
// status for its sender. A piece already taken or held is accepted again: it
// is a retry.
func (x *xfer) put(ctx context.Context, i int, data []byte) (int, string) {
if !x.await(ctx, xferWait, func() bool { return x.err != "" || x.done || i < x.low+xferBuffer }) {
return http.StatusServiceUnavailable, "нет места, повторите"
}
defer x.mu.Unlock()
if x.err != "" || x.done {
return http.StatusGone, x.gone()
}
if i < 0 || i >= x.n || int64(len(data)) != x.pieceLen(i) {
return http.StatusBadRequest, "не тот кусок"
}
if i >= x.low && x.chunks[i] == nil {
x.chunks[i] = data
x.notify()
}
return http.StatusNoContent, ""
}
// take hands over piece i, the next one in order, and lets it go.
func (x *xfer) take(ctx context.Context, i int, d time.Duration) ([]byte, error) {
if !x.await(ctx, d, func() bool { return x.err != "" || x.chunks[i] != nil }) {
return nil, errors.New("кусок не пришёл вовремя")
}
defer x.mu.Unlock()
if x.err != "" {
return nil, errors.New(x.err)
}
data := x.chunks[i]
delete(x.chunks, i)
x.low = i + 1
x.notify()
return data, nil
}
func (x *xfer) gone() string {
if x.err != "" {
return x.err
}
return "передача закончена"
}
// Read makes an upload a plain reader, for an agent that takes a file whole.
type xferReader struct {
ctx context.Context
x *xfer
i int
buf []byte
}
func (r *xferReader) Read(p []byte) (int, error) {
for len(r.buf) == 0 {
if r.i >= r.x.n {
return 0, io.EOF
}
data, err := r.x.take(r.ctx, r.i, xferIdle)
if err != nil {
return 0, err
}
r.buf, r.i = data, r.i+1
}
n := copy(p, r.buf)
r.buf = r.buf[n:]
return n, nil
}
type xfers struct {
mu sync.Mutex
m map[string]*xfer
}
func (c *xfers) open(agentID string, size int64) *xfer {
x := &xfer{
id: randToken()[:24], agentID: agentID, size: -1,
changed: make(chan struct{}), chunks: map[int][]byte{},
sized: make(chan struct{}), last: time.Now(),
}
if size >= 0 {
x.size, x.n = size, pieces(size)
close(x.sized)
}
c.mu.Lock()
if c.m == nil {
c.m = map[string]*xfer{}
}
// Forget the ones left behind: a closed tab, a client gone for good.
for id, old := range c.m {
old.mu.Lock()
stale := time.Since(old.last) > xferIdle
old.mu.Unlock()
if stale {
old.fail("передача брошена")
delete(c.m, id)
}
}
c.m[x.id] = x
c.mu.Unlock()
return x
}
func (c *xfers) get(id string) *xfer {
c.mu.Lock()
defer c.mu.Unlock()
return c.m[id]
}
// forAgent returns the transfer if it is the asking agent's.
func (c *xfers) forAgent(id, agentID string) *xfer {
if x := c.get(id); x != nil && x.agentID == agentID {
return x
}
return nil
}
func (c *xfers) drop(id string) {
c.mu.Lock()
delete(c.m, id)
c.mu.Unlock()
}
// waitTask runs a task on the agent and waits for its result, or for ctx to
// end.
func (s *Server) waitTask(ctx context.Context, agentID, typ string, args any, timeout time.Duration) (proto.TaskResult, error) {
raw, _ := json.Marshal(args)
taskID, ch := s.hub.CreateTask(agentID, typ, raw)
defer s.hub.DiscardTask(taskID)
t := time.NewTimer(timeout)
defer t.Stop()
select {
case res := <-ch:
return res, nil
case <-ctx.Done():
return proto.TaskResult{}, ctx.Err()
case <-t.C:
return proto.TaskResult{}, errors.New("таймаут ожидания агента")
}
}
// unknownOp tells an agent that predates the task.
func unknownOp(res proto.TaskResult) bool {
return !res.OK && strings.Contains(res.Error, "неизвестная операция")
}
// fsAgent answers the request itself and returns false when the agent is
// unknown or offline.
func (s *Server) fsAgent(w http.ResponseWriter, agentID string, asJSON bool) bool {
a, err := s.db.GetAgent(agentID)
if err != nil || a == nil {
http.Error(w, "not found", http.StatusNotFound)
return false
}
if !s.online(a) {
if asJSON {
writeJSON(w, proto.TaskResult{Error: "машина офлайн"})
} else {
http.Error(w, "машина офлайн", http.StatusServiceUnavailable)
}
return false
}
return true
}
// readPiece reads a piece's body into a buffer of its own size, so pieces
// leave no trail of grown buffers behind.
func readPiece(w http.ResponseWriter, r *http.Request) ([]byte, error) {
if r.ContentLength >= 0 && r.ContentLength <= xferChunk {
data := make([]byte, r.ContentLength)
_, err := io.ReadFull(r.Body, data)
return data, err
}
return io.ReadAll(http.MaxBytesReader(w, r.Body, xferChunk))
}
// ---------- upload: browser -> agent ----------
// POST /api/panel/agents/{id}/fs/upload?path=...&size=N - start writing a
// file. The answer names the transfer and the piece size; the browser then
// PUTs the pieces and asks for the result.
func (s *Server) handleFSUploadStart(w http.ResponseWriter, r *http.Request) {
s.uploadStart(w, r, proto.TaskFSPut, "")
}
// uploadStart starts a file upload to the agent's task (fs.put on the
// machine, sshfs.put onto an SSH host through it, conn naming the
// connection).
func (s *Server) uploadStart(w http.ResponseWriter, r *http.Request, task, conn string) {
agentID, p := r.PathValue("id"), r.URL.Query().Get("path")
if !s.fsAgent(w, agentID, true) {
return
}
size, err := strconv.ParseInt(r.URL.Query().Get("size"), 10, 64)
if p == "" || err != nil || size < 0 {
writeJSON(w, proto.TaskResult{Error: "не указан путь или размер"})
return
}
if size > proto.FSFileCap {
writeJSON(w, proto.TaskResult{Error: "файл больше 512 МБ"})
return
}
x := s.xfers.open(agentID, size)
go s.runUpload(x, p, task, conn)
data, _ := json.Marshal(map[string]any{"id": x.id, "chunk": xferChunk})
writeJSON(w, proto.TaskResult{OK: true, Data: data})
}
// runUpload has the agent write the file and keeps the result for the
// browser to collect.
func (s *Server) runUpload(x *xfer, p, task, conn string) {
ctx, cancel := context.WithTimeout(context.Background(), xferTimeout)
defer cancel()
args := proto.FSXferArgs{Path: p, Xfer: x.id, Size: x.size, Chunk: xferChunk, Conn: conn}
res, err := s.waitTask(ctx, x.agentID, task, args, xferTimeout)
if err == nil && unknownOp(res) && task != proto.TaskFSPut {
res = proto.TaskResult{Error: "агент не умеет файлы по SSH - обновите его"}
} else if err == nil && unknownOp(res) {
res = s.legacyUpload(ctx, x, p)
}
if err != nil {
res = proto.TaskResult{Error: err.Error()}
}
if !res.OK {
x.fail(res.Error)
}
x.finish(res)
// The result waits a while for the browser to come for it.
time.AfterFunc(xferIdle, func() { s.xfers.drop(x.id) })
}
// legacyUpload writes a file through an agent that predates the pieces.
func (s *Server) legacyUpload(ctx context.Context, x *xfer, p string) proto.TaskResult {
if x.size > fsLegacyCap {
return proto.TaskResult{Error: "клиент принимает файлы до 32 МБ - обновите его"}
}
data, err := io.ReadAll(&xferReader{ctx: ctx, x: x})
if err != nil {
return proto.TaskResult{Error: err.Error()}
}
args := proto.FSWriteArgs{Path: p, Data: base64.StdEncoding.EncodeToString(data)}
res, err := s.waitTask(ctx, x.agentID, proto.TaskFSWrite, args, fsTaskTimeout)
if err != nil {
return proto.TaskResult{Error: err.Error()}
}
return res
}
// PUT /api/panel/fs/xfer/{xid}/{i} - piece i of a file being uploaded. 503:
// no room yet, send it again; 410: the transfer is over.
func (s *Server) handleXferPut(w http.ResponseWriter, r *http.Request) {
x := s.xfers.get(r.PathValue("xid"))
i, err := strconv.Atoi(r.PathValue("i"))
if x == nil || err != nil {
http.Error(w, "передача не найдена", http.StatusGone)
return
}
data, err := readPiece(w, r)
if err != nil {
http.Error(w, "кусок не дошёл", http.StatusBadRequest)
return
}
code, msg := x.put(r.Context(), i, data)
if code == http.StatusNoContent {
w.WriteHeader(code)
return
}
http.Error(w, msg, code)
}
// POST /api/panel/fs/xfer/{xid}/cancel - the browser gives the upload up.
func (s *Server) handleXferCancel(w http.ResponseWriter, r *http.Request) {
if x := s.xfers.get(r.PathValue("xid")); x != nil {
x.fail("отменено")
}
w.WriteHeader(http.StatusNoContent)
}
// GET /api/panel/fs/xfer/{xid}/result - the upload's result, once the agent
// has written the file; {"done":false} means ask again.
func (s *Server) handleXferResult(w http.ResponseWriter, r *http.Request) {
x := s.xfers.get(r.PathValue("xid"))
if x == nil {
writeJSON(w, map[string]any{"done": true, "result": proto.TaskResult{Error: "передача не найдена"}})
return
}
if !x.await(r.Context(), xferWait, func() bool { return x.done }) {
writeJSON(w, map[string]any{"done": false})
return
}
res := x.result
x.mu.Unlock()
writeJSON(w, map[string]any{"done": true, "result": res})
}
// GET /api/agent/fs/xfer/{xid}/{i}?ack=K - piece i of the upload, for the
// agent to write. K says the agent has written every piece below it, so
// those are let go. 503: not here yet, ask again; 410: the transfer is over.
func (s *Server) handleAgentXferGet(w http.ResponseWriter, r *http.Request) {
a := s.authenticateAgent(r)
if a == nil {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
x := s.xfers.forAgent(r.PathValue("xid"), a.ID)
i, err := strconv.Atoi(r.PathValue("i"))
if x == nil || err != nil {
http.Error(w, "передача не найдена", http.StatusGone)
return
}
if ack, err := strconv.Atoi(r.URL.Query().Get("ack")); err == nil {
x.mu.Lock()
if ack > x.low {
for k := x.low; k < ack; k++ {
delete(x.chunks, k)
}
x.low = ack
x.notify()
}
x.mu.Unlock()
}
if !x.await(r.Context(), xferWait, func() bool { return x.err != "" || x.chunks[i] != nil }) {
http.Error(w, "кусок ещё не пришёл", http.StatusServiceUnavailable)
return
}
if x.err != "" {
msg := x.err
x.mu.Unlock()
http.Error(w, msg, http.StatusGone)
return
}
data := x.chunks[i]
x.mu.Unlock()
w.Header().Set("Content-Type", "application/octet-stream")
w.Header().Set("Content-Length", strconv.Itoa(len(data)))
_, _ = w.Write(data)
}
// ---------- download: agent -> browser ----------
// inlineTypes are the files the panel shows in place rather than downloads.
var inlineTypes = map[string]string{
".png": "image/png", ".jpg": "image/jpeg", ".jpeg": "image/jpeg", ".gif": "image/gif",
".webp": "image/webp", ".bmp": "image/bmp", ".ico": "image/x-icon", ".avif": "image/avif",
".svg": "image/svg+xml",
".mp4": "video/mp4", ".m4v": "video/mp4", ".webm": "video/webm", ".ogv": "video/ogg", ".mov": "video/quicktime",
".mp3": "audio/mpeg", ".wav": "audio/wav", ".ogg": "audio/ogg", ".oga": "audio/ogg", ".m4a": "audio/mp4",
".aac": "audio/aac", ".flac": "audio/flac", ".opus": "audio/ogg",
".pdf": "application/pdf",
}
// fileHeaders sets what the browser needs to download or show the file. A
// file shown in place runs nothing: no scripts in an SVG, whatever it holds.
// A PDF is the exception, as the browser's viewer will not open in a sandbox;
// it runs in that viewer, not in the panel's page.
func fileHeaders(w http.ResponseWriter, p string, inline bool) {
name := path.Base(p)
ctype := "application/octet-stream"
if t, ok := inlineTypes[strings.ToLower(path.Ext(name))]; ok && inline {
ctype = t
}
disp := "attachment"
if inline {
disp = "inline"
}
w.Header().Set("Content-Type", ctype)
w.Header().Set("Content-Disposition", disp+"; filename=\""+strings.ReplaceAll(name, "\"", "")+"\"")
w.Header().Set("X-Content-Type-Options", "nosniff")
if ctype != "application/pdf" {
w.Header().Set("Content-Security-Policy", "sandbox; default-src 'none'; img-src data:; style-src 'unsafe-inline'")
}
w.Header().Set("Cache-Control", "no-store")
}
// GET /api/panel/agents/{id}/fs/download?path=...[&inline=1] - the file, as
// one response fed by the pieces the agent sends. inline=1 serves pictures,
// video and sound for the page to show.
func (s *Server) handleFSDownload(w http.ResponseWriter, r *http.Request) {
s.download(w, r, proto.TaskFSSend, "")
}
// download sends a file from the agent's task (fs.send, or sshfs.send for
// an SSH host through it) as one response.
func (s *Server) download(w http.ResponseWriter, r *http.Request, task, conn string) {
agentID, p := r.PathValue("id"), r.URL.Query().Get("path")
inline := r.URL.Query().Get("inline") != ""
if !s.fsAgent(w, agentID, false) {
return
}
x := s.xfers.open(agentID, -1)
defer s.xfers.drop(x.id)
defer x.fail("браузер ушёл") // stops the agent if the response ends early
raw, _ := json.Marshal(proto.FSXferArgs{Path: p, Xfer: x.id, Size: -1, Chunk: xferChunk, Conn: conn})
taskID, ch := s.hub.CreateTask(agentID, task, raw)
defer s.hub.DiscardTask(taskID)
// The agent's result: an error ends the transfer wherever it stands.
results := make(chan proto.TaskResult, 1)
go func() {
select {
case res := <-ch:
if !res.OK && !unknownOp(res) {
x.fail(res.Error)
}
results <- res
case <-r.Context().Done():
}
}()
select {
case <-x.sized:
case res := <-results:
if unknownOp(res) && task != proto.TaskFSSend {
http.Error(w, "агент не умеет файлы по SSH - обновите его", http.StatusBadGateway)
return
}
if unknownOp(res) {
s.legacyDownload(w, r, agentID, p, inline)
return
}
msg := res.Error
if msg == "" {
msg = "агент не отдал файл"
}
http.Error(w, msg, http.StatusBadGateway)
return
case <-r.Context().Done():
return
case <-time.After(fsTaskTimeout):
http.Error(w, "таймаут ожидания агента", http.StatusGatewayTimeout)
return
}
fileHeaders(w, p, inline)
w.Header().Set("Content-Length", strconv.FormatInt(x.size, 10))
rc := http.NewResponseController(w)
for i := 0; i < x.n; i++ {
data, err := x.take(r.Context(), i, xferIdle)
if err == nil {
_, err = w.Write(data)
}
if err != nil {
// Cut the response off, so the browser does not take a part of
// the file for all of it.
x.fail(err.Error())
panic(http.ErrAbortHandler)
}
_ = rc.Flush()
}
}
// legacyDownload gets a file from an agent that predates the pieces.
func (s *Server) legacyDownload(w http.ResponseWriter, r *http.Request, agentID, p string, inline bool) {
res, err := s.waitTask(r.Context(), agentID, proto.TaskFSDownload, proto.FSPathArgs{Path: p}, fsTaskTimeout)
if err != nil {
http.Error(w, err.Error(), http.StatusGatewayTimeout)
return
}
if !res.OK {
http.Error(w, res.Error, http.StatusBadGateway)
return
}
var rr proto.FSReadResult
if err := json.Unmarshal(res.Data, &rr); err != nil {
http.Error(w, "bad result", http.StatusBadGateway)
return
}
data, _ := base64.StdEncoding.DecodeString(rr.Content)
fileHeaders(w, p, inline)
if rr.Truncated {
w.Header().Set("X-Truncated", "1")
}
_, _ = w.Write(data)
}
// POST /api/agent/fs/xfer/{xid}/{i} - piece i of the file the browser asked
// for, X-Size its whole size. 503: no room yet, send it again; 410: the
// browser is gone.
func (s *Server) handleAgentXferPost(w http.ResponseWriter, r *http.Request) {
a := s.authenticateAgent(r)
if a == nil {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
x := s.xfers.forAgent(r.PathValue("xid"), a.ID)
i, err := strconv.Atoi(r.PathValue("i"))
if x == nil || err != nil {
http.Error(w, "передача не найдена", http.StatusGone)
return
}
size, err := strconv.ParseInt(r.Header.Get("X-Size"), 10, 64)
if err != nil || size < 0 || size > proto.FSFileCap || !x.setSize(size) {
http.Error(w, "не тот размер", http.StatusBadRequest)
return
}
data, err := readPiece(w, r)
if err != nil {
http.Error(w, "кусок не дошёл", http.StatusBadRequest)
return
}
code, msg := x.put(r.Context(), i, data)
if code == http.StatusNoContent {
w.WriteHeader(code)
return
}
http.Error(w, msg, code)
}
+39 -7
View File
@@ -16,7 +16,7 @@ func writeJSON(w http.ResponseWriter, v any) {
_ = json.NewEncoder(w).Encode(v)
}
// POST /api/agent/enroll — exchange the shared enroll secret for a per-agent id+token.
// POST /api/agent/enroll - exchange the shared enroll secret for a per-agent id+token.
func (s *Server) handleEnroll(w http.ResponseWriter, r *http.Request) {
secret := r.Header.Get("X-Enroll-Secret")
if !constantTimeEqual(secret, s.db.EnrollSecret()) {
@@ -36,7 +36,7 @@ func (s *Server) handleEnroll(w http.ResponseWriter, r *http.Request) {
writeJSON(w, proto.EnrollResponse{AgentID: id, Token: token})
}
// GET /api/agent/poll — control channel. Held until a job is queued or timeout.
// GET /api/agent/poll - control channel. Held until a job is queued or timeout.
func (s *Server) handlePoll(w http.ResponseWriter, r *http.Request) {
a := s.authenticateAgent(r)
if a == nil {
@@ -45,6 +45,7 @@ func (s *Server) handlePoll(w http.ResponseWriter, r *http.Request) {
}
// The agent sends its version and OS user occasionally; record them when
// present. Agents older than 0.0.2 send no user, so keep what we have.
s.viaDirectA.Store(a.ID, s.viaDirect(r))
v := r.Header.Get("X-Agent-Version")
u := firstNonEmpty(r.Header.Get("X-Agent-User"), a.RunUser)
if v != "" && (v != a.Version || u != a.RunUser) {
@@ -52,10 +53,41 @@ func (s *Server) handlePoll(w http.ResponseWriter, r *http.Request) {
} else {
_ = s.db.TouchAgent(a.ID)
}
writeJSON(w, s.hub.WaitControl(a.ID, pollHold))
msg := s.hub.WaitControl(r.Context().Done(), a.ID, pollHold)
// Set after the hold so a setting changed meanwhile reaches the agent now.
s.setTunnelHeader(w)
w.Header().Set(proto.TransportHeader, s.transportOf(a.ID))
writeJSON(w, msg)
}
// GET /api/agent/task/{id} — the agent fetches the task to run.
// tunnelAddrHeader carries the direct-tunnel address ("host:port", host empty
// = the panel's host) to agents, on every poll and on /api/agent/config.
const tunnelAddrHeader = "X-Tunnel-Addr"
func (s *Server) setTunnelHeader(w http.ResponseWriter) {
if addr := s.tunnelAdvert(); addr != "" {
w.Header().Set(tunnelAddrHeader, addr)
}
if d := s.db.DirectAddress(); d != "" {
w.Header().Set(proto.FallbackHeader, d)
w.Header().Set(proto.MainHeader, s.db.HostAddress())
}
}
// GET /api/agent/config - settings an agent picks up when it connects, without
// waiting out a poll hold.
func (s *Server) handleAgentConfig(w http.ResponseWriter, r *http.Request) {
a := s.authenticateAgent(r)
if a == nil {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
s.setTunnelHeader(w)
w.Header().Set(proto.TransportHeader, s.transportOf(a.ID))
w.WriteHeader(http.StatusNoContent)
}
// GET /api/agent/task/{id} - the agent fetches the task to run.
func (s *Server) handleTaskFetch(w http.ResponseWriter, r *http.Request) {
if s.authenticateAgent(r) == nil {
http.Error(w, "unauthorized", http.StatusUnauthorized)
@@ -69,7 +101,7 @@ func (s *Server) handleTaskFetch(w http.ResponseWriter, r *http.Request) {
writeJSON(w, env)
}
// POST /api/agent/task/{id}/result — the agent returns the task result.
// POST /api/agent/task/{id}/result - the agent returns the task result.
func (s *Server) handleTaskResult(w http.ResponseWriter, r *http.Request) {
if s.authenticateAgent(r) == nil {
http.Error(w, "unauthorized", http.StatusUnauthorized)
@@ -84,7 +116,7 @@ func (s *Server) handleTaskResult(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusNoContent)
}
// GET /api/agent/term/{sid}/in — downstream (panel -> agent) long-poll.
// GET /api/agent/term/{sid}/in - downstream (panel -> agent) long-poll.
func (s *Server) handleTermIn(w http.ResponseWriter, r *http.Request) {
if s.authenticateAgent(r) == nil {
http.Error(w, "unauthorized", http.StatusUnauthorized)
@@ -103,7 +135,7 @@ func (s *Server) handleTermIn(w http.ResponseWriter, r *http.Request) {
writeJSON(w, proto.InBatch{Frames: frames})
}
// POST /api/agent/term/{sid}/out — upstream (agent PTY -> panel).
// POST /api/agent/term/{sid}/out - upstream (agent PTY -> panel).
func (s *Server) handleTermOut(w http.ResponseWriter, r *http.Request) {
if s.authenticateAgent(r) == nil {
http.Error(w, "unauthorized", http.StatusUnauthorized)
+315 -45
View File
@@ -5,11 +5,13 @@ import (
"errors"
"fmt"
"log"
"net"
"net/http"
"net/url"
"slices"
"strconv"
"strings"
"sync"
"time"
"github.com/x-grk/longpolling-remote-access/internal/proto"
@@ -86,7 +88,12 @@ func (s *Server) handleLoginSubmit(w http.ResponseWriter, r *http.Request) {
admin, _ := s.db.AdminByUsername(username)
if admin == nil || !checkPassword(admin.PassHash, pw) {
s.limiter.fin(ip, false)
s.render(w, "login.html", map[string]any{"Error": "Неверный логин или пароль."})
s.render(w, "login.html", map[string]any{"Error": "Неверный логин или пароль.", "Username": username})
return
}
// An admin who turned the password off signs in with a passkey only.
if s.db.PasswordOff(admin.ID) {
s.render(w, "login.html", map[string]any{"Error": "Для этого аккаунта вход только по ключу доступа.", "Username": username})
return
}
s.limiter.fin(ip, true)
@@ -105,38 +112,69 @@ func (s *Server) handleLogout(w http.ResponseWriter, r *http.Request) {
// --- dashboard ---
func (s *Server) handleDashboard(w http.ResponseWriter, r *http.Request) {
agents, err := s.db.ListAgents()
if err != nil {
http.Error(w, "internal error", http.StatusInternalServerError)
return
s.renderPage(w, r, "dashboard", s.dashboardData(r))
}
// dashboardData is the machines list as the user may see it.
func (s *Server) dashboardData(r *http.Request) map[string]any {
agents, _ := s.db.ListAgents()
type row struct {
Agent
Online bool
Seen string
}
me := s.currentAdmin(r)
rows := make([]row, 0, len(agents))
for _, a := range agents {
rows = append(rows, row{Agent: a, Online: a.Online(), Seen: humanSince(a.LastSeen)})
if !s.db.CanUseAgent(me, a.ID) {
continue
}
s.render(w, "dashboard.html", map[string]any{
"Admin": s.currentAdmin(r),
on := s.online(&a)
rows = append(rows, row{Agent: a, Online: on, Seen: seenText(on, a.LastSeen)})
}
return map[string]any{
"Admin": me,
"IsAdmin": me.IsAdmin(),
"Agents": rows,
"CSRF": csrfToken(r),
})
}
}
func (s *Server) handleTerminalPage(w http.ResponseWriter, r *http.Request) {
id := r.PathValue("id")
a, err := s.db.GetAgent(id)
if err != nil || a == nil {
http.NotFound(w, r)
// seenText is the "activity" cell: a connected agent is active now, whatever
// the time of its last recorded poll.
func seenText(online bool, lastSeen time.Time) string {
if online {
return "сейчас"
}
return humanSince(lastSeen)
}
// GET /api/panel/agents/presence - who is online, for the dashboard to keep
// its status column live without reloading.
func (s *Server) handlePresence(w http.ResponseWriter, r *http.Request) {
agents, err := s.db.ListAgents()
if err != nil {
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
s.render(w, "terminal.html", map[string]any{
"Agent": a,
"CSRF": csrfToken(r),
})
type item struct {
ID string `json:"id"`
Name string `json:"name"`
Hostname string `json:"hostname"`
User string `json:"user"` // who a terminal opens as
Online bool `json:"online"`
Seen string `json:"seen"`
}
me := s.currentAdmin(r)
out := make([]item, 0, len(agents))
for _, a := range agents {
if !s.db.CanUseAgent(me, a.ID) {
continue
}
on := s.online(&a)
out = append(out, item{ID: a.ID, Name: a.Name, Hostname: a.Hostname, User: a.TerminalUser(), Online: on, Seen: seenText(on, a.LastSeen)})
}
writeJSON(w, out)
}
func (s *Server) handleAgentDelete(w http.ResponseWriter, r *http.Request) {
@@ -145,7 +183,16 @@ func (s *Server) handleAgentDelete(w http.ResponseWriter, r *http.Request) {
return
}
id := r.PathValue("id")
// Its forward rules go with it: left behind they would keep their host
// ports bound and busy with no page to remove them from.
rules, _ := s.db.ListForwardRulesByAgent(id)
for _, rule := range rules {
s.removeForward(rule.ID)
}
_ = s.db.DeleteAgent(id)
s.stats.drop(id)
s.relays.drop(id, nil)
s.hub.Forget(id)
http.Redirect(w, r, "/", http.StatusSeeOther)
}
@@ -158,22 +205,44 @@ func (s *Server) handleAgentEdit(w http.ResponseWriter, r *http.Request) {
}
data := map[string]any{
"Agent": a,
"IsAdmin": s.currentAdmin(r).IsAdmin(),
"CSRF": csrfToken(r),
"ServerVersion": Version,
"Outdated": a.Version != "" && a.Version != Version,
"Updated": r.URL.Query().Get("updated") == "1",
"UpdateMsg": r.URL.Query().Get("umsg"),
"UpdateErr": r.URL.Query().Get("uperr"),
"UserOK": r.URL.Query().Get("uok"),
"UserErr": r.URL.Query().Get("uerr"),
"Transport": firstNonEmpty(a.Transport, proto.TransportAuto),
"OnWS": s.wsAgents.On(a.ID),
"ViaDirect": viaDirectOf(&s.viaDirectA, a.ID),
"Online": s.online(a),
}
// The user list lives on the machine, so it is only known while it is online.
if a.Online() {
if s.online(a) {
if users, err := s.agentUsers(id); err == nil {
data["Users"] = users
} else {
data["UsersErr"] = err.Error()
}
}
s.render(w, "edit.html", data)
s.renderPage(w, r, "edit", data)
}
// POST /agents/{id}/transport - how the agent is to reach the server: auto,
// ws or lp. A connected agent picks it up within a poll round.
func (s *Server) handleAgentTransport(w http.ResponseWriter, r *http.Request) {
if !checkCSRF(r) {
http.Error(w, "bad csrf", http.StatusForbidden)
return
}
id, t := r.PathValue("id"), r.FormValue("transport")
switch t {
case proto.TransportAuto, proto.TransportWS, proto.TransportLP:
_ = s.db.SetAgentTransport(id, t)
}
http.Redirect(w, r, "/agents/"+id+"/edit", http.StatusSeeOther)
}
// usersTaskTimeout bounds how long the edit page waits for the agent.
@@ -187,7 +256,7 @@ func (s *Server) agentUsers(agentID string) (*proto.SysUsersResult, error) {
case res := <-ch:
if !res.OK {
// Agents older than 0.0.2 don't know this task.
return nil, errors.New("клиент не поддерживает выбор пользователя — обновите его")
return nil, errors.New("клиент не поддерживает выбор пользователя - обновите его")
}
var out proto.SysUsersResult
if err := json.Unmarshal(res.Data, &out); err != nil {
@@ -217,8 +286,8 @@ func (s *Server) handleAgentSetUser(w http.ResponseWriter, r *http.Request) {
return
}
name := strings.TrimSpace(r.FormValue("user"))
if !a.Online() {
editUserRedirect(w, r, id, "uerr", "Машина офлайн — сменить пользователя нельзя.")
if !s.online(a) {
editUserRedirect(w, r, id, "uerr", "Машина офлайн - сменить пользователя нельзя.")
return
}
// Only accept a user the machine itself reports, so a typo (or an agent too
@@ -258,8 +327,8 @@ func (s *Server) handleAgentAddUser(w http.ResponseWriter, r *http.Request) {
http.NotFound(w, r)
return
}
if !a.Online() {
editUserRedirect(w, r, id, "uerr", "Машина офлайн — создать пользователя нельзя.")
if !s.online(a) {
editUserRedirect(w, r, id, "uerr", "Машина офлайн - создать пользователя нельзя.")
return
}
name := strings.TrimSpace(r.FormValue("name"))
@@ -299,17 +368,28 @@ func (s *Server) handleAgentUpdate(w http.ResponseWriter, r *http.Request) {
http.NotFound(w, r)
return
}
if !a.Online() {
http.Redirect(w, r, "/agents/"+id+"/edit?updated=0", http.StatusSeeOther)
back := func(q url.Values) { http.Redirect(w, r, "/agents/"+id+"/edit?"+q.Encode(), http.StatusSeeOther) }
if !s.online(a) {
back(url.Values{"uperr": {"машина не в сети"}})
return
}
taskID, ch := s.hub.CreateTask(id, proto.TaskSysUpdate, nil)
defer s.hub.DiscardTask(taskID)
select {
case <-ch:
case <-time.After(30 * time.Second):
if admin := s.currentAdmin(r); admin != nil {
s.db.Audit(admin.ID, id, "update:"+a.Version+" -> "+Version)
}
// The download can take a while on a slow link; under a proxy's timeout.
res, err := s.waitTask(r.Context(), id, proto.TaskSysUpdate, nil, 80*time.Second)
switch {
case err != nil:
back(url.Values{"uperr": {"агент не ответил: " + err.Error()}})
case !res.OK:
back(url.Values{"uperr": {res.Error}})
default:
var d struct {
Status string `json:"status"`
}
_ = json.Unmarshal(res.Data, &d)
back(url.Values{"updated": {"1"}, "umsg": {d.Status}})
}
http.Redirect(w, r, "/agents/"+id+"/edit?updated=1", http.StatusSeeOther)
}
func (s *Server) handleAgentRename(w http.ResponseWriter, r *http.Request) {
@@ -322,45 +402,152 @@ func (s *Server) handleAgentRename(w http.ResponseWriter, r *http.Request) {
if name != "" {
_ = s.db.RenameAgent(id, name)
}
http.Redirect(w, r, "/", http.StatusSeeOther)
// Back to «Изменить», with the new name: the machine dialog stays open.
http.Redirect(w, r, "/agents/"+id+"/edit", http.StatusSeeOther)
}
// --- settings ---
func (s *Server) handleSettings(w http.ResponseWriter, r *http.Request) {
admins, _ := s.db.ListAdmins()
keyCounts, passOff, agentCounts := map[int64]int{}, map[int64]bool{}, map[int64]int{}
for _, a := range admins {
keyCounts[a.ID] = s.db.CountAdminKeys(a.ID)
passOff[a.ID] = s.db.PasswordOff(a.ID)
agentCounts[a.ID] = len(s.db.UserAgents(a.ID))
}
allAgents, _ := s.db.ListAgents()
host := s.db.HostAddress()
direct := s.db.DirectAddress()
secret := s.db.EnrollSecret()
code := s.db.InstallCode()
ufwInstalled, ufwActive, ufwRunnable := s.ufw.status()
ufwRules, _ := s.db.ListUfwRules()
s.render(w, "settings.html", map[string]any{
dbNow, dbChosen := s.db.Backend()
s.renderPage(w, r, "settings", map[string]any{
"Admin": s.currentAdmin(r),
"Host": host,
"Direct": direct,
"DirectShort": fmt.Sprintf("wget -qO- %s/i/%s | sudo sh", direct, code),
"DirectCurl": fmt.Sprintf("curl -fsSL %s/install.sh | sudo HOST=%s FALLBACK_HOST=%s ENROLL_SECRET=%s sh", direct, direct, host, secret),
"Secret": secret,
"CodeLen": s.db.InstallCodeLen(),
"Admins": admins,
"KeyCounts": keyCounts,
"AgentCounts": agentCounts,
"AllAgents": allAgents,
"Me": s.currentAdmin(r),
"PassOff": passOff,
"CSRF": csrfToken(r),
"InstallShort": fmt.Sprintf("wget -qO- %s/i/%s | sudo sh", host, code),
"InstallCurl": fmt.Sprintf("curl -fsSL %s/install.sh | sudo HOST=%s ENROLL_SECRET=%s sh", host, host, secret),
"InstallWget": fmt.Sprintf("wget -qO- %s/install.sh | sudo HOST=%s ENROLL_SECRET=%s sh", host, host, secret),
"ServerVersion": Version,
"AIModels": s.aiModels(),
"Search": s.searchConf(),
"ImageKey": func() string { _, k := s.imageConf(); return k }(),
"DBNow": dbNow,
"DBChosen": dbChosen,
"UfwInstalled": ufwInstalled,
"UfwActive": ufwActive,
"UfwRunnable": ufwRunnable,
"UfwEnabled": s.db.UfwEnabled(),
"UfwRules": ufwRules,
"UfwError": s.ufw.LastError(),
"TunnelPort": s.TunnelPort(),
"TunnelDefault": s.tunnelDefault,
"TunnelHost": s.db.TunnelHost(),
"TunnelCustom": s.db.TunnelPort() > 0,
"Error": map[string]string{
"tunnel_host": "Некорректный хост туннеля: нужен IP или имя без схемы и порта.",
"host": "Некорректный адрес: нужен URL вида https://panel.example.com, без пути.",
"tunnel_port": "Некорректный порт туннеля.",
"ufw_port": "Некорректное правило ufw: нужен порт 1-65535 и протокол tcp или udp.",
"ufw_off": "Правило ufw не добавлено: ufw выключен или интеграция с ним отключена.",
"ufw_failed": "Правило ufw не добавлено: команда ufw завершилась ошибкой.",
"ufw_exists": "Этот порт уже открыт в ufw не панелью - правило осталось как было, панель его не отслеживает.",
"tunnel_listen": "Не удалось занять новый порт туннеля на хосте (возможно, уже используется). Порт не изменён.",
}[r.URL.Query().Get("err")],
})
}
// POST /settings/tunnel - host and port agents use for the direct tunnel.
func (s *Server) handleSetTunnel(w http.ResponseWriter, r *http.Request) {
if !checkCSRF(r) {
http.Error(w, "bad csrf", http.StatusForbidden)
return
}
if s.tunnelDefault == 0 {
http.Redirect(w, r, "/settings", http.StatusSeeOther)
return
}
host := strings.TrimSpace(r.FormValue("tunnel_host"))
host = strings.TrimSuffix(strings.TrimPrefix(host, "["), "]")
if !validTunnelHost(host) {
http.Redirect(w, r, "/settings?err=tunnel_host", http.StatusSeeOther)
return
}
// An empty port goes back to the one the server was started with.
port, custom := s.tunnelDefault, ""
if v := strings.TrimSpace(r.FormValue("tunnel_port")); v != "" {
n, err := strconv.Atoi(v)
if err != nil || n < 1 || n > 65535 {
http.Redirect(w, r, "/settings?err=tunnel_port", http.StatusSeeOther)
return
}
port = n
if n != s.tunnelDefault {
custom = v
}
}
if err := s.setTunnelPort(port); err != nil {
log.Printf("tunnel: move to port %d: %v", port, err)
http.Redirect(w, r, "/settings?err=tunnel_listen", http.StatusSeeOther)
return
}
_ = s.db.SetConfig(cfgTunnelPort, custom)
_ = s.db.SetConfig(cfgTunnelHost, host)
http.Redirect(w, r, "/settings", http.StatusSeeOther)
}
// validTunnelHost accepts an empty value, an IP, or a bare host name.
func validTunnelHost(host string) bool {
if host == "" || net.ParseIP(host) != nil {
return true
}
if len(host) > 253 {
return false
}
for _, c := range host {
switch {
case c >= 'a' && c <= 'z', c >= 'A' && c <= 'Z', c >= '0' && c <= '9', c == '-', c == '.', c == '_':
default:
return false
}
}
return true
}
func (s *Server) handleSetHost(w http.ResponseWriter, r *http.Request) {
if !checkCSRF(r) {
http.Error(w, "bad csrf", http.StatusForbidden)
return
}
host := strings.TrimRight(strings.TrimSpace(r.FormValue("host")), "/")
direct := strings.TrimRight(strings.TrimSpace(r.FormValue("direct")), "/")
for _, v := range []string{host, direct} {
if v == "" {
continue
}
if u, err := url.Parse(v); err != nil || (u.Scheme != "http" && u.Scheme != "https") || u.Host == "" || (u.Path != "" && u.Path != "/") {
http.Redirect(w, r, "/settings?err=host", http.StatusSeeOther)
return
}
}
if host != "" {
_ = s.db.SetConfig(cfgHostAddress, host)
}
_ = s.db.SetConfig(cfgDirectAddr, direct)
http.Redirect(w, r, "/settings", http.StatusSeeOther)
}
@@ -371,7 +558,22 @@ func (s *Server) handleRotateSecret(w http.ResponseWriter, r *http.Request) {
}
_ = s.db.SetConfig(cfgEnrollSecret, randToken())
// Rotate the short install code too so old /i/<code> links stop working.
_ = s.db.SetConfig(cfgInstallCode, shortCode())
_ = s.db.SetConfig(cfgInstallCode, shortCode(s.db.InstallCodeLen()))
http.Redirect(w, r, "/settings", http.StatusSeeOther)
}
// POST /settings/install/code - a new install code, of the length given:
// the old /i/<code> link stops working.
func (s *Server) handleInstallCode(w http.ResponseWriter, r *http.Request) {
if !checkCSRF(r) {
http.Error(w, "bad csrf", http.StatusForbidden)
return
}
if n, err := strconv.Atoi(r.FormValue("len")); err == nil {
n = min(installCodeMax, max(installCodeMin, n))
_ = s.db.SetConfig(cfgInstallLen, strconv.Itoa(n))
}
_ = s.db.SetConfig(cfgInstallCode, shortCode(s.db.InstallCodeLen()))
http.Redirect(w, r, "/settings", http.StatusSeeOther)
}
@@ -382,12 +584,21 @@ func (s *Server) handleAddAdmin(w http.ResponseWriter, r *http.Request) {
}
username := strings.TrimSpace(r.FormValue("username"))
pw := r.FormValue("password")
role := RoleAdmin
if r.FormValue("role") == RoleUser {
role = RoleUser
}
if username != "" && len(pw) >= 8 {
if hash, err := hashPassword(pw); err == nil {
_ = s.db.CreateAdmin(username, hash)
if id, err := s.db.CreateUser(username, hash, role); err == nil && role == RoleUser {
_ = s.db.SetUserAgents(id, r.Form["agents"])
}
if me := s.currentAdmin(r); me != nil {
s.db.Audit(me.ID, "", "user_add:"+username+":"+role)
}
}
http.Redirect(w, r, "/settings", http.StatusSeeOther)
}
http.Redirect(w, r, "/settings#users", http.StatusSeeOther)
}
func (s *Server) handleDeleteAdmin(w http.ResponseWriter, r *http.Request) {
@@ -397,7 +608,7 @@ func (s *Server) handleDeleteAdmin(w http.ResponseWriter, r *http.Request) {
}
id, _ := strconv.ParseInt(r.FormValue("id"), 10, 64)
// Never delete the last admin.
if n, _ := s.db.CountAdmins(); n > 1 {
if u, _ := s.db.AdminByID(id); u != nil && (!u.IsAdmin() || s.db.CountRole(RoleAdmin) > 1) {
_ = s.db.DeleteAdmin(id)
}
http.Redirect(w, r, "/settings", http.StatusSeeOther)
@@ -408,10 +619,33 @@ func (s *Server) handleUfwToggle(w http.ResponseWriter, r *http.Request) {
http.Error(w, "bad csrf", http.StatusForbidden)
return
}
_ = s.db.SetUfwEnabled(r.FormValue("enabled") == "1")
on := r.FormValue("enabled") == "1"
_ = s.db.SetUfwEnabled(on)
if on {
// Open what forwards created while the integration was off need.
s.ufw.sync()
}
http.Redirect(w, r, "/settings", http.StatusSeeOther)
}
func (s *Server) handleUfwAdd(w http.ResponseWriter, r *http.Request) {
if !checkCSRF(r) {
http.Error(w, "bad csrf", http.StatusForbidden)
return
}
port, _ := strconv.Atoi(strings.TrimSpace(r.FormValue("port")))
proto := r.FormValue("proto")
if port < 1 || port > 65535 || (proto != "tcp" && proto != "udp") {
http.Redirect(w, r, "/settings?err=ufw_port", http.StatusSeeOther)
return
}
target := "/settings"
if key := s.ufw.allowManual(port, proto); key != "" {
target += "?err=" + key
}
http.Redirect(w, r, target, http.StatusSeeOther)
}
func (s *Server) handleUfwRevoke(w http.ResponseWriter, r *http.Request) {
if !checkCSRF(r) {
http.Error(w, "bad csrf", http.StatusForbidden)
@@ -427,7 +661,7 @@ func (s *Server) handleUfwRevoke(w http.ResponseWriter, r *http.Request) {
// handleInstallScript serves the generic installer; the secret is NOT baked in,
// so it must be supplied via env or the interactive prompt.
func (s *Server) handleInstallScript(w http.ResponseWriter, r *http.Request) {
s.renderInstall(w, "")
s.renderInstall(w, r, "")
}
// handleShortInstall serves the installer behind the short /i/<code> link with
@@ -435,20 +669,48 @@ func (s *Server) handleInstallScript(w http.ResponseWriter, r *http.Request) {
//
// wget -qO- https://host/i/<code> | sudo sh
func (s *Server) handleShortInstall(w http.ResponseWriter, r *http.Request) {
// A short code can be guessed: wrong ones count against the address
// like wrong passwords do.
ip := clientIP(r)
if s.limiter.blocked(ip) {
http.Error(w, "too many attempts", http.StatusTooManyRequests)
return
}
code := r.PathValue("code")
want := s.db.InstallCode()
if want == "" || !constantTimeEqual(code, want) {
s.limiter.fin(ip, false)
http.NotFound(w, r)
return
}
s.renderInstall(w, s.db.EnrollSecret())
s.renderInstall(w, r, s.db.EnrollSecret())
}
// renderInstall writes install.sh with the current host and an optional secret.
func (s *Server) renderInstall(w http.ResponseWriter, secret string) {
// viaDirect reports whether the request came to the direct address.
func (s *Server) viaDirect(r *http.Request) bool {
d := directHost(s.db.DirectAddress())
if d == "" {
return false
}
host := r.Host
if h, _, err := net.SplitHostPort(host); err == nil {
host = h
}
return strings.EqualFold(host, d)
}
// renderInstall writes install.sh with the server's addresses and an
// optional secret. Fetched from the direct address, it makes that the
// agent's address and the main one its fallback; otherwise the other way.
func (s *Server) renderInstall(w http.ResponseWriter, r *http.Request, secret string) {
host, fallback := s.db.HostAddress(), s.db.DirectAddress()
if s.viaDirect(r) {
host, fallback = fallback, host
}
w.Header().Set("Content-Type", "text/x-shellscript; charset=utf-8")
if err := s.installTmpl.Execute(w, map[string]any{
"Host": s.db.HostAddress(),
"Host": host,
"Fallback": fallback,
"Secret": secret,
}); err != nil {
log.Printf("install.sh render: %v", err)
@@ -461,3 +723,11 @@ func (s *Server) downloadHandler() http.Handler {
// install one-liner works before the machine is enrolled.
return http.FileServer(http.Dir(s.distDir))
}
// viaDirectOf tells whether the agent last reached the server at the direct
// address.
func viaDirectOf(m *sync.Map, id string) bool {
v, _ := m.Load(id)
b, _ := v.(bool)
return b
}
+32 -55
View File
@@ -1,11 +1,9 @@
package server
import (
"encoding/base64"
"encoding/json"
"io"
"net/http"
"path"
"time"
"github.com/x-grk/longpolling-remote-access/internal/proto"
@@ -22,22 +20,22 @@ var fsOps = map[string]string{
"unzip": proto.TaskFSUnzip,
"zip": proto.TaskFSZip,
"move": proto.TaskFSMove,
"stat": proto.TaskFSStat,
"chmod": proto.TaskFSChmod,
"copy": proto.TaskFSCopy,
"db.tables": proto.TaskDBTables,
"db.rows": proto.TaskDBRows,
"db.set": proto.TaskDBSet,
"db.add": proto.TaskDBAdd,
"db.del": proto.TaskDBDel,
"db.sql": proto.TaskDBSQL,
"termcwd": proto.TaskTermCwd,
}
// fsTaskTimeout bounds how long we wait for an agent to answer a file op.
const fsTaskTimeout = 60 * time.Second
func (s *Server) handleFilesPage(w http.ResponseWriter, r *http.Request) {
id := r.PathValue("id")
a, err := s.db.GetAgent(id)
if err != nil || a == nil {
http.NotFound(w, r)
return
}
s.render(w, "files.html", map[string]any{"Agent": a})
}
// POST /api/panel/agents/{id}/fs/{op} — run a file-manager operation on the
// POST /api/panel/agents/{id}/fs/{op} - run a file-manager operation on the
// agent and return its result. The request body is the op's JSON args and is
// forwarded verbatim to the agent.
func (s *Server) handleFSOp(w http.ResponseWriter, r *http.Request) {
@@ -53,7 +51,7 @@ func (s *Server) handleFSOp(w http.ResponseWriter, r *http.Request) {
http.NotFound(w, r)
return
}
if !a.Online() {
if !s.online(a) {
writeJSON(w, proto.TaskResult{Error: "машина офлайн"})
return
}
@@ -63,6 +61,26 @@ func (s *Server) handleFSOp(w http.ResponseWriter, r *http.Request) {
return
}
// Changes to a database go to the audit log, with what was done.
if op == "db.set" || op == "db.add" || op == "db.del" || op == "db.sql" {
var d struct {
Path, Table, Column, SQL string
}
_ = json.Unmarshal(args, &d)
what := d.Path + " " + d.Table
if d.Column != "" {
what += "." + d.Column
}
if d.SQL != "" {
what += " " + d.SQL
}
if len(what) > 300 {
what = what[:300]
}
if admin := s.currentAdmin(r); admin != nil {
s.db.Audit(admin.ID, agentID, op+":"+what)
}
}
taskID, resultCh := s.hub.CreateTask(agentID, taskType, args)
defer s.hub.DiscardTask(taskID)
@@ -73,44 +91,3 @@ func (s *Server) handleFSOp(w http.ResponseWriter, r *http.Request) {
writeJSON(w, proto.TaskResult{Error: "таймаут ожидания агента"})
}
}
// GET /api/panel/agents/{id}/fs/download?path=... — stream a file as a download.
func (s *Server) handleFSDownload(w http.ResponseWriter, r *http.Request) {
agentID := r.PathValue("id")
p := r.URL.Query().Get("path")
a, err := s.db.GetAgent(agentID)
if err != nil || a == nil {
http.NotFound(w, r)
return
}
if !a.Online() {
http.Error(w, "машина офлайн", http.StatusServiceUnavailable)
return
}
args, _ := json.Marshal(proto.FSPathArgs{Path: p})
taskID, ch := s.hub.CreateTask(agentID, proto.TaskFSDownload, args)
defer s.hub.DiscardTask(taskID)
select {
case res := <-ch:
if !res.OK {
http.Error(w, res.Error, http.StatusBadGateway)
return
}
var rr proto.FSReadResult
if err := json.Unmarshal(res.Data, &rr); err != nil {
http.Error(w, "bad result", http.StatusBadGateway)
return
}
data, _ := base64.StdEncoding.DecodeString(rr.Content)
name := path.Base(p)
w.Header().Set("Content-Type", "application/octet-stream")
w.Header().Set("Content-Disposition", "attachment; filename=\""+name+"\"")
if rr.Truncated {
w.Header().Set("X-Truncated", "1")
}
_, _ = w.Write(data)
case <-time.After(fsTaskTimeout):
http.Error(w, "таймаут ожидания агента", http.StatusGatewayTimeout)
}
}
+54 -7
View File
@@ -3,6 +3,7 @@ package server
import (
"log"
"net/http"
"net/url"
"strconv"
"strings"
)
@@ -17,18 +18,37 @@ func (s *Server) handleForwardsPage(w http.ResponseWriter, r *http.Request) {
rules, _ := s.db.ListForwardRulesByAgent(id)
errMsg := map[string]string{
"busy": "Этот порт хоста уже занят другим правилом.",
"tunnel": "Этот порт занят прямым туннелем панели (см. Настройки).",
"port": "Некорректный номер порта.",
"listen": "Не удалось занять порт на хосте (возможно, уже используется).",
"db": "Ошибка базы данных.",
}[r.URL.Query().Get("err")]
s.render(w, "forwards.html", map[string]any{
s.renderPage(w, r, "forwards", map[string]any{
"Agent": a,
"Rules": rules,
"CSRF": csrfToken(r),
"Error": errMsg,
"TunnelPort": s.TunnelPort(),
"PanelPort": panelPort(s.db.HostAddress()),
"UfwEnabled": s.ufw.enabled(),
})
}
// panelPort is the port of the panel URL agents dial, for the mode hints.
func panelPort(host string) string {
u, err := url.Parse(host)
if err != nil {
return ""
}
if p := u.Port(); p != "" {
return p
}
if u.Scheme == "http" {
return "80"
}
return "443"
}
// POST /agents/{id}/forwards/add
func (s *Server) handleForwardAdd(w http.ResponseWriter, r *http.Request) {
if !checkCSRF(r) {
@@ -57,12 +77,17 @@ func (s *Server) handleForwardAdd(w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, "/agents/"+id+"/forwards?err=port", http.StatusSeeOther)
return
}
// Reject a host port already bound by another forward rule.
// Reject a host port already bound by another forward rule or the tunnel.
if hostPort == s.TunnelPort() {
http.Redirect(w, r, "/agents/"+id+"/forwards?err=tunnel", http.StatusSeeOther)
return
}
if s.db.HostPortUsed(hostPort) {
http.Redirect(w, r, "/agents/"+id+"/forwards?err=busy", http.StatusSeeOther)
return
}
rule := ForwardRule{AgentID: id, ClientPort: clientPort, HostPort: hostPort, ListenAddr: listenAddr, Proto: protoKind}
rule := ForwardRule{AgentID: id, ClientPort: clientPort, HostPort: hostPort, ListenAddr: listenAddr, Proto: protoKind,
NoUfw: r.FormValue("no_ufw") != "", Label: forwardLabel(r.FormValue("label"))}
ruleID, err := s.db.CreateForwardRule(rule)
if err != nil {
http.Redirect(w, r, "/agents/"+id+"/forwards?err=db", http.StatusSeeOther)
@@ -89,13 +114,35 @@ func (s *Server) handleForwardDelete(w http.ResponseWriter, r *http.Request) {
}
rid, _ := strconv.ParseInt(r.PathValue("rid"), 10, 64)
rule, _ := s.db.GetForwardRule(rid)
s.fwd.close(rid)
_ = s.db.DeleteForwardRule(rid)
// Remove the ufw rule bound to this forward rule (shared 8446 stays).
s.ufw.revokeForRule(rid)
if rule == nil || !s.db.CanUseAgent(s.currentAdmin(r), rule.AgentID) {
http.NotFound(w, r)
return
}
s.removeForward(rid)
dest := "/"
if rule != nil {
dest = "/agents/" + rule.AgentID + "/forwards"
}
http.Redirect(w, r, dest, http.StatusSeeOther)
}
// removeForward closes a forward rule's listener and deletes the rule, along
// with the ufw rule bound to it (the shared tunnel port stays while another
// direct forward needs it).
func (s *Server) removeForward(rid int64) {
s.fwd.close(rid)
_ = s.db.DeleteForwardRule(rid)
s.ufw.revokeForRule(rid)
}
// forwardLabelMax bounds a rule's label, in characters.
const forwardLabelMax = 80
// forwardLabel tidies the optional note of a rule: one line, bounded length.
func forwardLabel(v string) string {
v = strings.Join(strings.Fields(v), " ")
if r := []rune(v); len(r) > forwardLabelMax {
v = string(r[:forwardLabelMax])
}
return v
}
+542
View File
@@ -0,0 +1,542 @@
package server
import (
"crypto/rand"
"encoding/json"
"errors"
"io"
"net/http"
"strconv"
"strings"
"sync"
"time"
"github.com/x-grk/longpolling-remote-access/internal/proto"
)
const (
// statTaskTimeout bounds how long a status request waits for the agent.
statTaskTimeout = 8 * time.Second
// statCacheFor is how long one sample answers every open status page, so
// several tabs cost the agent a single reading.
statCacheFor = 2 * time.Second
// speedWindow is how long the agent is asked to measure each direction
// for. An agent that predates the argument keeps to its own 3 s.
speedWindow = 7 * time.Second
// speedTaskTimeout is how long a speed test may take. One that is not done
// by then is answered with what the server has counted itself.
speedTaskTimeout = 35 * time.Second
// speedDownCap bounds one download stream; speedDownFor ends it if the
// agent does not hang up first.
speedDownCap = 1 << 30
speedDownFor = 2 * speedWindow
speedUpCap = 64 << 20
// speedRelayUpFor is how long the agent's upload is passed on to the
// browser: the agent's own window with slack.
speedRelayUpFor = speedWindow + time.Second
)
// statCache holds the last sample per agent and serializes its refresh.
type statCache struct {
mu sync.Mutex
m map[string]*statEntry
}
type statEntry struct {
mu sync.Mutex
at time.Time
res proto.TaskResult
}
func (c *statCache) entry(agentID string) *statEntry {
c.mu.Lock()
defer c.mu.Unlock()
if c.m == nil {
c.m = map[string]*statEntry{}
}
e := c.m[agentID]
if e == nil {
e = &statEntry{}
c.m[agentID] = e
}
return e
}
func (c *statCache) drop(agentID string) {
c.mu.Lock()
delete(c.m, agentID)
c.mu.Unlock()
}
func (s *Server) handleStatusPage(w http.ResponseWriter, r *http.Request) {
a, err := s.db.GetAgent(r.PathValue("id"))
if err != nil || a == nil {
http.NotFound(w, r)
return
}
s.renderPage(w, r, "status", map[string]any{"Agent": a, "RelayUpMs": speedRelayUpFor.Milliseconds()})
}
// runTask runs a one-shot task on the agent and waits for its result.
func (s *Server) runTask(agentID, typ string, args []byte, timeout time.Duration) (proto.TaskResult, error) {
taskID, ch := s.hub.CreateTask(agentID, typ, args)
defer s.hub.DiscardTask(taskID)
t := time.NewTimer(timeout)
defer t.Stop()
select {
case res := <-ch:
return res, nil
case <-t.C:
return proto.TaskResult{}, errors.New("таймаут ожидания клиента")
}
}
// onlineAgent answers the request itself and returns false when the agent is
// unknown or offline.
func (s *Server) onlineAgent(w http.ResponseWriter, r *http.Request) (string, bool) {
id := r.PathValue("id")
a, err := s.db.GetAgent(id)
if err != nil || a == nil {
http.NotFound(w, r)
return "", false
}
if !s.online(a) {
writeJSON(w, proto.TaskResult{Error: "машина офлайн"})
return "", false
}
return id, true
}
// oldAgentError replaces the "unknown task" answer of an agent that predates
// the task.
func oldAgentError(res proto.TaskResult) proto.TaskResult {
if !res.OK && strings.Contains(res.Error, "неизвестная операция") {
res.Error = "клиент не поддерживает эту функцию - обновите его (нужен 0.0.5+)"
}
return res
}
// GET /api/panel/agents/{id}/stat - CPU, RAM and disk of the machine right now.
func (s *Server) handleAgentStat(w http.ResponseWriter, r *http.Request) {
id, ok := s.onlineAgent(w, r)
if !ok {
return
}
e := s.stats.entry(id)
e.mu.Lock()
defer e.mu.Unlock()
if time.Since(e.at) > statCacheFor {
res, err := s.runTask(id, proto.TaskSysStat, nil, statTaskTimeout)
if err != nil {
// Cached too, so pages queued behind this one don't each wait it out.
res = proto.TaskResult{Error: err.Error()}
}
e.res, e.at = oldAgentError(res), time.Now()
}
writeJSON(w, e.res)
}
// POST /api/panel/agents/{id}/speed - have the agent measure its link to us.
// With relay=1 its streams run through the relay opened for it, to and from
// the browser.
func (s *Server) handleAgentSpeed(w http.ResponseWriter, r *http.Request) {
id, ok := s.onlineAgent(w, r)
if !ok {
return
}
relay := r.URL.Query().Get("relay") != ""
m := s.meters.start(id)
defer s.meters.stop(id, m)
if relay {
rl := s.relays.get(id)
if rl == nil {
writeJSON(w, proto.TaskResult{Error: "замер через браузер не открыт"})
return
}
defer s.relays.drop(id, rl)
}
args, _ := json.Marshal(proto.SysSpeedArgs{Relay: relay, WindowMs: int(speedWindow.Milliseconds())})
res, err := s.runTask(id, proto.TaskSysSpeed, args, speedTaskTimeout)
if err != nil {
part, ok := m.result()
if !ok {
writeJSON(w, proto.TaskResult{Error: err.Error()})
return
}
part.Relay = relay
data, _ := json.Marshal(part)
res = proto.TaskResult{OK: true, Data: data}
}
res = oldAgentError(res)
if relay && res.OK {
// An agent that predates the relay ignores the argument and measures
// its own link.
var d proto.SysSpeedResult
if json.Unmarshal(res.Data, &d) != nil || !d.Relay {
res = proto.TaskResult{Error: "клиент не умеет замер через браузер - обновите его (нужен 0.0.6+)"}
}
}
writeJSON(w, res)
}
// speedMeter counts what flows through the streams of one speed test as the
// server sees it: for the live reading on the page, and for the result of a
// test that ran out of time.
type speedMeter struct {
mu sync.Mutex
start time.Time
phase string // "ping", "down", "up"
pingAt time.Time
pingMs float64
down speedCount
up speedCount
}
// speedCount is the bytes of one direction and when they went through. The
// first piece only starts the clock.
type speedCount struct {
n, skip int64
first, last time.Time
}
func (c *speedCount) add(n int) {
now := time.Now()
if c.first.IsZero() {
c.first, c.skip = now, int64(n)
}
c.n += int64(n)
c.last = now
}
func (c *speedCount) bps() float64 {
d := c.last.Sub(c.first).Seconds()
if d <= 0 {
return 0
}
return float64(c.n-c.skip) * 8 / d
}
// ping notes one of the agent's ping probes: they come one right after
// another, so the gap between two is its round trip.
func (m *speedMeter) ping() {
if m == nil {
return
}
m.mu.Lock()
now := time.Now()
if !m.pingAt.IsZero() {
if ms := float64(now.Sub(m.pingAt).Microseconds()) / 1000; m.pingMs == 0 || ms < m.pingMs {
m.pingMs = ms
}
}
m.pingAt = now
m.mu.Unlock()
}
func (m *speedMeter) add(up bool, n int) {
if m == nil || n <= 0 {
return
}
m.mu.Lock()
if up {
m.phase = "up"
m.up.add(n)
} else {
m.phase = "down"
m.down.add(n)
}
m.mu.Unlock()
}
// result is the test as counted so far; false with nothing measured yet.
func (m *speedMeter) result() (proto.SysSpeedResult, bool) {
m.mu.Lock()
defer m.mu.Unlock()
res := proto.SysSpeedResult{PingMs: m.pingMs, DownBps: m.down.bps(), UpBps: m.up.bps(), Partial: true}
return res, res.DownBps > 0
}
// meterWriter counts what it passes on; a nil meter counts nothing.
type meterWriter struct {
w io.Writer
m *speedMeter
up bool
}
func (c meterWriter) Write(p []byte) (int, error) {
n, err := c.w.Write(p)
c.m.add(c.up, n)
return n, err
}
// speedMeters holds the meter of each agent under test.
type speedMeters struct {
mu sync.Mutex
m map[string]*speedMeter
}
func (c *speedMeters) start(agentID string) *speedMeter {
m := &speedMeter{start: time.Now(), phase: "ping"}
c.mu.Lock()
if c.m == nil {
c.m = map[string]*speedMeter{}
}
c.m[agentID] = m
c.mu.Unlock()
return m
}
func (c *speedMeters) get(agentID string) *speedMeter {
c.mu.Lock()
defer c.mu.Unlock()
return c.m[agentID]
}
// stop forgets the agent's meter if it is still m.
func (c *speedMeters) stop(agentID string, m *speedMeter) {
c.mu.Lock()
if c.m[agentID] == m {
delete(c.m, agentID)
}
c.mu.Unlock()
}
// GET /api/panel/agents/{id}/speed/progress - the bytes counted so far in the
// agent's running speed test, for the page to turn into a live rate. An empty
// phase means no test is running.
func (s *Server) handleSpeedProgress(w http.ResponseWriter, r *http.Request) {
var out struct {
Phase string `json:"phase"`
Down int64 `json:"down"`
Up int64 `json:"up"`
Ms int64 `json:"ms"` // since the test began, by the server's clock
}
if m := s.meters.get(r.PathValue("id")); m != nil {
m.mu.Lock()
out.Phase, out.Down, out.Up = m.phase, m.down.n, m.up.n
out.Ms = time.Since(m.start).Milliseconds()
m.mu.Unlock()
}
w.Header().Set("Cache-Control", "no-store")
writeJSON(w, out)
}
// speedBlock is one chunk of incompressible bytes, made once and sent over and
// over, so a speed test costs the server no CPU beyond the copy.
var speedBlock = sync.OnceValue(func() []byte {
b := make([]byte, 64<<10)
rand.Read(b)
return b
})
func speedStreamHeaders(w http.ResponseWriter) {
w.Header().Set("Content-Type", "application/octet-stream")
w.Header().Set("Cache-Control", "no-store")
// Keep a proxy in front from buffering or compressing the stream.
w.Header().Set("X-Accel-Buffering", "no")
}
// GET /api/agent/speed/down?bytes=N - stream N bytes (default: as much as fits
// in the time window) for the agent to time. bytes=0 is the ping probe.
func (s *Server) handleSpeedDown(w http.ResponseWriter, r *http.Request) {
a := s.authenticateAgent(r)
if a == nil {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
m := s.meters.get(a.ID)
left := speedDownCap
if v := r.URL.Query().Get("bytes"); v != "" {
if n, err := strconv.Atoi(v); err == nil && n >= 0 && n < left {
left = n
}
}
if left == 0 {
m.ping()
}
speedStreamHeaders(w)
block := speedBlock()
deadline := time.Now().Add(speedDownFor)
for left > 0 && time.Now().Before(deadline) && r.Context().Err() == nil {
chunk := block
if len(chunk) > left {
chunk = chunk[:left]
}
n, err := w.Write(chunk)
if err != nil {
return
}
m.add(false, n)
left -= n
}
}
// POST /api/agent/speed/up - swallow the agent's upload.
func (s *Server) handleSpeedUp(w http.ResponseWriter, r *http.Request) {
a := s.authenticateAgent(r)
if a == nil {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
sink := meterWriter{w: io.Discard, m: s.meters.get(a.ID), up: true}
_, _ = io.Copy(sink, http.MaxBytesReader(w, r.Body, speedUpCap))
w.WriteHeader(http.StatusNoContent)
}
// speedRelay joins the agent's speed test streams to the browser's, so the
// bytes travel browser - panel - client. Both pipes are unbuffered: a slow
// reader holds its writer back, and the rate is that of the whole path.
type speedRelay struct {
downR *io.PipeReader // browser -> agent
downW *io.PipeWriter
upR *io.PipeReader // agent -> browser
upW *io.PipeWriter
}
func (rl *speedRelay) close() {
rl.downR.CloseWithError(io.ErrClosedPipe)
rl.downW.CloseWithError(io.ErrClosedPipe)
rl.upR.CloseWithError(io.ErrClosedPipe)
rl.upW.CloseWithError(io.ErrClosedPipe)
}
// speedRelays holds the relay of each agent under test, one per agent.
type speedRelays struct {
mu sync.Mutex
m map[string]*speedRelay
}
// open starts a fresh relay for the agent, ending the one it had.
func (c *speedRelays) open(agentID string) {
rl := &speedRelay{}
rl.downR, rl.downW = io.Pipe()
rl.upR, rl.upW = io.Pipe()
c.mu.Lock()
if c.m == nil {
c.m = map[string]*speedRelay{}
}
old := c.m[agentID]
c.m[agentID] = rl
c.mu.Unlock()
if old != nil {
old.close()
}
}
func (c *speedRelays) get(agentID string) *speedRelay {
c.mu.Lock()
defer c.mu.Unlock()
return c.m[agentID]
}
// drop ends the agent's relay; with rl given, only if it is still that one.
func (c *speedRelays) drop(agentID string, rl *speedRelay) {
c.mu.Lock()
cur := c.m[agentID]
if cur != nil && (rl == nil || rl == cur) {
delete(c.m, agentID)
} else {
cur = nil
}
c.mu.Unlock()
if cur != nil {
cur.close()
}
}
// GET /api/panel/ping - an empty answer for the browser to time.
func (s *Server) handlePanelPing(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Cache-Control", "no-store")
w.WriteHeader(http.StatusNoContent)
}
// POST /api/panel/agents/{id}/speed/relay - open the relay of a speed test
// that runs through the browser. The test itself ends it.
func (s *Server) handleSpeedRelayOpen(w http.ResponseWriter, r *http.Request) {
id, ok := s.onlineAgent(w, r)
if !ok {
return
}
s.relays.open(id)
writeJSON(w, proto.TaskResult{OK: true})
}
// POST /api/panel/agents/{id}/speed/down - a piece of the browser's upload,
// passed on to the agent. 204 asks for the next piece, 200 says the agent has
// stopped reading.
func (s *Server) handleSpeedRelayFeed(w http.ResponseWriter, r *http.Request) {
body := http.MaxBytesReader(w, r.Body, speedUpCap)
rl := s.relays.get(r.PathValue("id"))
if rl != nil {
if _, err := io.Copy(rl.downW, body); err == nil {
w.WriteHeader(http.StatusNoContent)
return
}
}
// Take the rest of the piece before answering: an answer sent over an
// unread body makes Go drop the connection, which a proxy in front turns
// into a 502.
_, _ = io.Copy(io.Discard, body)
w.Header().Set("Cache-Control", "no-store")
_, _ = io.WriteString(w, "done")
}
// GET /api/panel/agents/{id}/speed/up - the agent's upload, for the browser to
// time. Ends when the agent is done sending.
func (s *Server) handleSpeedRelayDrain(w http.ResponseWriter, r *http.Request) {
id := r.PathValue("id")
rl := s.relays.get(id)
if rl == nil {
http.Error(w, "gone", http.StatusGone)
return
}
speedStreamHeaders(w)
_, _ = io.Copy(meterWriter{w: w, m: s.meters.get(id), up: true}, rl.upR)
// The browser may have left: don't let the agent keep sending to nobody.
rl.upR.CloseWithError(io.ErrClosedPipe)
}
// GET /api/agent/speed/relay/down - what the browser uploads, for the agent to
// time.
func (s *Server) handleSpeedRelayDown(w http.ResponseWriter, r *http.Request) {
a := s.authenticateAgent(r)
if a == nil {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
rl := s.relays.get(a.ID)
if rl == nil {
http.NotFound(w, r)
return
}
speedStreamHeaders(w)
// The agent waits on the headers, the browser may not have sent a byte yet.
_ = http.NewResponseController(w).Flush()
_, _ = io.Copy(meterWriter{w: w, m: s.meters.get(a.ID)}, rl.downR)
// The agent hung up at the end of its window: stop the browser's upload.
rl.downR.CloseWithError(io.ErrClosedPipe)
}
// POST /api/agent/speed/relay/up - the agent's upload, passed on to the browser.
func (s *Server) handleSpeedRelayUp(w http.ResponseWriter, r *http.Request) {
a := s.authenticateAgent(r)
if a == nil {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
rl := s.relays.get(a.ID)
if rl == nil {
http.NotFound(w, r)
return
}
body := http.MaxBytesReader(w, r.Body, speedUpCap)
// A proxy in front may hand over the whole upload at once: pass on only
// what the browser takes in the window, or the agent would wait for the
// rest to trickle through a slower link.
t := time.AfterFunc(speedRelayUpFor, func() { rl.upW.Close() })
_, _ = io.Copy(rl.upW, body)
t.Stop()
rl.upW.Close()
_, _ = io.Copy(io.Discard, body)
w.WriteHeader(http.StatusNoContent)
}
+132 -6
View File
@@ -1,9 +1,18 @@
package server
import (
"context"
"encoding/json"
"io"
"log"
"net"
"net/http"
"strconv"
"strings"
"time"
"github.com/coder/websocket"
"github.com/coder/websocket/wsjson"
"github.com/x-grk/longpolling-remote-access/internal/proto"
)
@@ -12,7 +21,10 @@ import (
// same transport the agent uses), so it works through any reverse proxy without
// WebSocket upgrade support.
// POST /api/panel/term/{agentID}/open — create a terminal session.
// POST /api/panel/term/{agentID}/open - create a terminal session: a shell
// on the agent, or with {"ssh": {...}} in the body a shell the agent opens
// by SSH on another host. {"dir": "/path"} starts the shell in that folder
// (an older agent, or a folder not there: at home).
func (s *Server) handlePanelOpen(w http.ResponseWriter, r *http.Request) {
agentID := r.PathValue("agentID")
agent, err := s.db.GetAgent(agentID)
@@ -21,14 +33,50 @@ func (s *Server) handlePanelOpen(w http.ResponseWriter, r *http.Request) {
http.NotFound(w, r)
return
}
sess := s.hub.CreateSession(agentID, agent.LoginUser)
var body struct {
SSH *proto.SSHTarget `json:"ssh"`
Dir string `json:"dir"`
}
if r.ContentLength != 0 {
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 64<<10)).Decode(&body); err != nil && err != io.EOF {
http.Error(w, "bad request", http.StatusBadRequest)
return
}
}
if err := s.fillSaved(s.currentAdmin(r).ID, body.SSH); err != nil {
writeJSON(w, map[string]any{"error": err.Error()})
return
}
if t := body.SSH; t != nil {
t.Host, t.User = strings.TrimSpace(t.Host), strings.TrimSpace(t.User)
if t.Port == 0 {
t.Port = 22
}
if t.Host == "" || t.User == "" || t.Port < 1 || t.Port > 65535 || strings.ContainsAny(t.Host+t.User, " \t\r\n/") {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusBadRequest)
_ = json.NewEncoder(w).Encode(map[string]string{"error": "нужны хост, порт 1-65535 и пользователь"})
return
}
sess := s.hub.CreateSSH(agentID, *t)
if admin := s.currentAdmin(r); admin != nil {
s.db.Audit(admin.ID, agentID, "open_ssh:"+t.User+"@"+net.JoinHostPort(t.Host, strconv.Itoa(t.Port)))
}
writeJSON(w, map[string]string{"session_id": sess.ID})
return
}
dir := body.Dir
if !strings.HasPrefix(dir, "/") || strings.ContainsAny(dir, "\x00\r\n") || len(dir) > 4096 {
dir = ""
}
sess := s.hub.CreateSession(agentID, agent.LoginUser, dir)
if admin := s.currentAdmin(r); admin != nil {
s.db.Audit(admin.ID, agentID, "open_terminal:"+agent.TerminalUser())
}
writeJSON(w, map[string]string{"session_id": sess.ID})
}
// GET /api/panel/term/{sid}/out — long-poll for PTY output.
// GET /api/panel/term/{sid}/out - long-poll for PTY output.
func (s *Server) handlePanelOut(w http.ResponseWriter, r *http.Request) {
sess := s.hub.Session(r.PathValue("sid"))
if sess == nil {
@@ -49,9 +97,12 @@ type panelInMsg struct {
D string `json:"d,omitempty"`
Cols uint16 `json:"cols,omitempty"`
Rows uint16 `json:"rows,omitempty"`
// Seq numbers the messages of a session from 1; 0 from a page that
// predates it.
Seq uint64 `json:"seq,omitempty"`
}
// POST /api/panel/term/{sid}/in — send keystrokes/resize to the agent PTY.
// POST /api/panel/term/{sid}/in - send keystrokes/resize to the agent PTY.
func (s *Server) handlePanelIn(w http.ResponseWriter, r *http.Request) {
sess := s.hub.Session(r.PathValue("sid"))
if sess == nil {
@@ -63,16 +114,91 @@ func (s *Server) handlePanelIn(w http.ResponseWriter, r *http.Request) {
http.Error(w, "bad request", http.StatusBadRequest)
return
}
takePanelIn(sess, m)
w.WriteHeader(http.StatusNoContent)
}
// takePanelIn passes the browser's keystrokes or resize on to the agent; a
// repeat of one already taken is dropped.
func takePanelIn(sess *Session, m panelInMsg) {
if m.Seq > 0 && !sess.TakeSeq(m.Seq) {
return
}
switch m.T {
case "i":
sess.PushIn(proto.InFrame{Type: proto.FrameStdin, Data: m.D})
case "r":
sess.PushIn(proto.InFrame{Type: proto.FrameResize, Cols: m.Cols, Rows: m.Rows})
}
w.WriteHeader(http.StatusNoContent)
}
// POST /api/panel/term/{sid}/close — tear the session down.
// GET /api/panel/term/{sid}/ws - the browser's terminal on a WebSocket: what
// the "in" requests send comes as messages, and output goes out as soon as
// it is there, in the "out" answers' shape. The page falls back to those
// requests when this does not connect.
func (s *Server) handlePanelTermWS(w http.ResponseWriter, r *http.Request) {
sess := s.hub.Session(r.PathValue("sid"))
if sess == nil {
http.Error(w, "closed", http.StatusGone)
return
}
// Terminal text packs well: deflate with the context kept between
// messages, for the larger ones (screens of output, not keystrokes).
conn, err := websocket.Accept(w, r, &websocket.AcceptOptions{CompressionMode: websocket.CompressionContextTakeover, CompressionThreshold: 256})
if err != nil {
return
}
conn.SetReadLimit(1 << 20)
defer conn.CloseNow()
ctx, cancel := context.WithCancel(r.Context())
defer cancel()
go func() {
defer cancel()
for ctx.Err() == nil {
msgs, alive := sess.PollOutUntil(ctx.Done(), pollHold)
if ctx.Err() != nil {
return
}
out := map[string]any{"frames": msgs}
if !alive {
out = map[string]any{"closed": true}
}
wctx, wcancel := context.WithTimeout(ctx, 30*time.Second)
err := wsjson.Write(wctx, conn, out)
wcancel()
if err != nil || !alive {
return
}
}
}()
go func() {
t := time.NewTicker(wsPing)
defer t.Stop()
for {
select {
case <-ctx.Done():
return
case <-t.C:
pctx, pcancel := context.WithTimeout(ctx, 15*time.Second)
err := conn.Ping(pctx)
pcancel()
if err != nil {
cancel()
return
}
}
}
}()
for {
var m panelInMsg
if err := wsjson.Read(ctx, conn, &m); err != nil {
return
}
takePanelIn(sess, m)
}
}
// POST /api/panel/term/{sid}/close - tear the session down.
func (s *Server) handlePanelClose(w http.ResponseWriter, r *http.Request) {
s.hub.CloseSession(r.PathValue("sid"))
w.WriteHeader(http.StatusNoContent)
+109 -8
View File
@@ -7,8 +7,9 @@ import (
"github.com/x-grk/longpolling-remote-access/internal/proto"
)
// Hub is the in-memory broker between agents (HTTP long-polling) and the panel.
// It carries three kinds of work to an agent through its single control poll:
// Hub is the in-memory broker between agents (on a WebSocket, or by HTTP
// long polls) and the panel. It carries three kinds of work to an agent
// through its single control channel:
// terminal sessions, port-forward streams, and one-shot tasks (file manager).
// Nothing here is persisted.
type Hub struct {
@@ -22,6 +23,12 @@ type Hub struct {
sessions map[string]*Session
tasks map[string]*taskEntry
// Presence: control polls held open right now per agent, and when each
// agent's last one ended. started is when this hub came up.
polls map[string]int
lastPoll map[string]time.Time
started time.Time
}
// taskEntry is a pending one-shot task awaiting the agent's result.
@@ -31,7 +38,7 @@ type taskEntry struct {
}
// Session is one live byte stream between the panel and an agent (a PTY terminal
// or a raw TCP forward — the hub treats both identically).
// or a raw TCP forward - the hub treats both identically).
type Session struct {
ID string
AgentID string
@@ -39,6 +46,7 @@ type Session struct {
mu sync.Mutex
inBuf []proto.InFrame
inSignal chan struct{} // size-1; pulsed when inBuf gains data
inSeq uint64 // number of the last browser message taken
out chan proto.OutMessage // agent -> panel
@@ -52,6 +60,9 @@ func NewHub() *Hub {
ctrlWaiters: map[string]chan struct{}{},
sessions: map[string]*Session{},
tasks: map[string]*taskEntry{},
polls: map[string]int{},
lastPoll: map[string]time.Time{},
started: time.Now(),
}
}
@@ -84,9 +95,16 @@ func (h *Hub) newSession(agentID string) *Session {
// CreateSession registers a terminal session and queues it for the agent. user
// is the OS user to run the shell as ("" = the agent's own user).
func (h *Hub) CreateSession(agentID, user string) *Session {
func (h *Hub) CreateSession(agentID, user, dir string) *Session {
s := h.newSession(agentID)
h.enqueueJob(agentID, proto.ControlMessage{Type: proto.CtrlOpenTerminal, SessionID: s.ID, User: user})
h.enqueueJob(agentID, proto.ControlMessage{Type: proto.CtrlOpenTerminal, SessionID: s.ID, User: user, Dir: dir})
return s
}
// CreateSSH registers a terminal session the agent opens by SSH to t.
func (h *Hub) CreateSSH(agentID string, t proto.SSHTarget) *Session {
s := h.newSession(agentID)
h.enqueueJob(agentID, proto.ControlMessage{Type: proto.CtrlOpenSSH, SessionID: s.ID, SSH: &t})
return s
}
@@ -97,9 +115,56 @@ func (h *Hub) CreateForward(agentID string, port int) *Session {
return s
}
// WaitControl blocks until a job is queued for agentID or timeout elapses.
func (h *Hub) WaitControl(agentID string, timeout time.Duration) proto.ControlMessage {
deadline := time.After(timeout)
// pollGrace is how long an agent stays online between two control polls: it
// re-polls at once, so this only has to cover a slow round trip or one retry.
const pollGrace = 10 * time.Second
// Online reports whether the agent is connected: it holds a control poll open
// or has just finished one. ok is false while that cannot be told yet (right
// after a start, before agents had time to come back).
func (h *Hub) Online(agentID string) (online, ok bool) {
h.mu.Lock()
defer h.mu.Unlock()
if h.polls[agentID] > 0 {
return true, true
}
if t, seen := h.lastPoll[agentID]; seen {
return time.Since(t) < pollGrace, true
}
return false, time.Since(h.started) > pollHold+pollGrace
}
// Forget drops what the hub keeps about a deleted agent.
func (h *Hub) Forget(agentID string) {
h.mu.Lock()
delete(h.jobs, agentID)
delete(h.ctrlWaiters, agentID)
delete(h.lastPoll, agentID)
h.mu.Unlock()
}
// WaitControl blocks until a job is queued for agentID, the timeout elapses
// or the agent hangs up (done). The agent counts as online meanwhile.
func (h *Hub) WaitControl(done <-chan struct{}, agentID string, timeout time.Duration) proto.ControlMessage {
h.mu.Lock()
h.polls[agentID]++
h.mu.Unlock()
defer func() {
h.mu.Lock()
if h.polls[agentID]--; h.polls[agentID] <= 0 {
delete(h.polls, agentID)
}
// A poll cut off by the agent is not a sign of life.
select {
case <-done:
default:
h.lastPoll[agentID] = time.Now()
}
h.mu.Unlock()
}()
t := time.NewTimer(timeout)
defer t.Stop()
deadline := t.C
for {
h.mu.Lock()
if q := h.jobs[agentID]; len(q) > 0 {
@@ -119,6 +184,9 @@ func (h *Hub) WaitControl(agentID string, timeout time.Duration) proto.ControlMe
case <-w:
case <-deadline:
return proto.ControlMessage{Type: proto.CtrlIdle}
case <-done:
// The job, if one comes, stays queued for the next poll.
return proto.ControlMessage{Type: proto.CtrlIdle}
}
}
}
@@ -191,6 +259,19 @@ func (s *Session) close() { s.closeOnce.Do(func() { close(s.closed) }) }
// Done returns a channel closed when the session ends.
func (s *Session) Done() <-chan struct{} { return s.closed }
// TakeSeq reports whether the browser message numbered seq is new, and takes
// note of it. The browser sends them one at a time in rising order and sends
// one again when it got no answer, so a number not above the last is a repeat.
func (s *Session) TakeSeq(seq uint64) bool {
s.mu.Lock()
defer s.mu.Unlock()
if seq <= s.inSeq {
return false
}
s.inSeq = seq
return true
}
// PushIn appends a downstream frame (panel -> agent) and wakes the agent poll.
func (s *Session) PushIn(f proto.InFrame) {
s.mu.Lock()
@@ -202,6 +283,18 @@ func (s *Session) PushIn(f proto.InFrame) {
}
}
// Unpoll puts frames taken by PollIn back in front, for frames that could not
// be delivered.
func (s *Session) Unpoll(frames []proto.InFrame) {
s.mu.Lock()
s.inBuf = append(append([]proto.InFrame{}, frames...), s.inBuf...)
s.mu.Unlock()
select {
case s.inSignal <- struct{}{}:
default:
}
}
// PollIn drains queued downstream frames, blocking up to timeout for the first.
func (s *Session) PollIn(timeout time.Duration) ([]proto.InFrame, bool) {
deadline := time.After(timeout)
@@ -239,10 +332,18 @@ func (s *Session) Out() <-chan proto.OutMessage { return s.out }
// PollOut drains queued upstream messages, blocking up to timeout for the first.
func (s *Session) PollOut(timeout time.Duration) ([]proto.OutMessage, bool) {
return s.PollOutUntil(nil, timeout)
}
// PollOutUntil is PollOut that also gives up when done closes, taking
// nothing: for a reader that has gone, so no output is lost on it.
func (s *Session) PollOutUntil(done <-chan struct{}, timeout time.Duration) ([]proto.OutMessage, bool) {
var batch []proto.OutMessage
select {
case m := <-s.out:
batch = append(batch, m)
case <-done:
return nil, true
case <-time.After(timeout):
return nil, true
case <-s.closed:
+302
View File
@@ -0,0 +1,302 @@
package server
import (
"bytes"
"context"
"encoding/base64"
"encoding/json"
"errors"
"io"
"mime/multipart"
"net/http"
"net/textproto"
"net/url"
"regexp"
"strconv"
"strings"
"time"
)
// Pictures for the assistant: the model asks for one in its answer - a
// ```image block, a new picture or an edit of a photo - and the panel makes
// it through the Vertex API (https://s2.x-grk.com/api/vertex/docs). The
// work is a job there: it is put, then asked about until it is done. The
// API key is the admins', set in Настройки and never sent to the browser.
const (
cfgImageKey = "ai_image_key"
cfgImageURL = "ai_image_url" // not in the settings: the service's address, for tests
// DefaultImageURL is where the Vertex API is.
DefaultImageURL = "https://s2.x-grk.com"
imageMaxBytes = 20 << 20 // what the service takes of one photo
)
// imageConf is the service's address and the key (none: pictures are off).
func (s *Server) imageConf() (base, key string) {
base, _ = s.db.GetConfig(cfgImageURL)
if base == "" {
base = DefaultImageURL
}
key, _ = s.db.GetConfig(cfgImageKey)
return strings.TrimRight(base, "/"), strings.TrimSpace(key)
}
// imagePrompt tells the model it can make pictures, and how - when there
// is a key.
func (s *Server) imagePrompt() string {
if _, key := s.imageConf(); key == "" {
return ""
}
return "\n\nКартинки. Ты умеешь рисовать и править фото: сам, по смыслу разговора, без просьбы «вызови инструмент» и без лишних вопросов, " +
"когда пользователь просит нарисовать, сгенерировать, изобразить, сделать картинку, арт, логотип, обои, иллюстрацию " +
"или изменить, дорисовать, убрать, заменить что-то на фото.\n" +
"```image\nописание картинки\n``` - создать новую. Опиши сцену, стиль, свет, детали; язык описания любой.\n" +
"```image edit\nчто изменить на фото\n``` - править фото пользователя: то, что приложено к его сообщению " +
"(приложено несколько - после edit назови файл: ```image edit имя.jpg```); фото нет - последнюю созданную картинку.\n" +
"В первой строке блока можно задать формат: 1:1 (по умолчанию), 16:9, 9:16, 3:2, 2:3, 4:3, 3:4, auto - и качество: " +
"fast (по умолчанию, 1 кредит), artist (5, лучше), 0.2plus (10, лучшее); повышай качество, только если пользователь просит. " +
"Например: ```image 16:9 artist```.\n" +
"Сервис по умолчанию сам улучшает описание (дописывает детали и стиль) - это хорошо для короткого запроса. " +
"Если пользователь дал готовое, точное описание (промпт целиком, логотип «ровно как сказано», текст на картинке, строгая композиция) " +
"или просит ничего не добавлять - добавь noenhance: ```image noenhance 3:2```. Для правки фото это не применяется.\n" +
"Картинку покажет сама панель: ссылку в ответ не вставляй, скажи коротко, что получилось, и предложи поправить. " +
"Делается она до пары минут. Одно действие за ответ."
}
// imageError says in Russian what an error code of the service means.
func imageError(code, msg string) string {
switch code {
case "unauthorized":
return "сервис не принял API-ключ - проверьте его в Настройки › ИИ-ассистент › Генерация изображений"
case "banned":
return "аккаунт сервиса картинок заблокирован"
case "insufficient_credits":
return "на аккаунте сервиса картинок не хватает кредитов"
case "daily_limit":
return "исчерпан дневной лимит картинок"
case "job_busy":
return "уже делается другая картинка - дождитесь её"
case "invalid_prompt":
return "сервис не принял описание (пустое или длиннее 4000 символов)"
case "invalid_image":
return "сервис не принял фото (нужен png, jpg или webp)"
case "image_too_large":
return "фото больше 20 МБ"
case "tokens_exhausted":
return "сервис картинок перегружен - попробуйте позже"
case "generation_failed", "edit_failed":
return "сервис не справился с картинкой (кредиты вернутся)"
case "source_missing":
return "исходное фото потерялось на сервисе - загрузите его заново"
}
if msg == "" {
msg = "сервис картинок ответил ошибкой"
}
return msg
}
var imageClient = &http.Client{Timeout: 90 * time.Second}
// imageCall sends the request to the service with the key and reads its
// JSON; an error is words for the user.
func imageCall(ctx context.Context, req *http.Request, key string, v any) error {
req = req.WithContext(ctx)
req.Header.Set("Authorization", "Bearer "+key)
req.Header.Set("Accept", "application/json")
resp, err := imageClient.Do(req)
if err != nil {
return errors.New("сервис картинок не отвечает: " + err.Error())
}
defer resp.Body.Close()
b, _ := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
var e struct {
OK *bool `json:"ok"`
ErrorCode string `json:"error_code"`
Error string `json:"error"`
}
if json.Unmarshal(b, &e) != nil {
return errors.New("сервис картинок ответил " + resp.Status)
}
if (e.OK != nil && !*e.OK) || resp.StatusCode/100 != 2 {
return errors.New(imageError(e.ErrorCode, e.Error))
}
return json.Unmarshal(b, v)
}
var (
imageRatios = map[string]bool{"1:1": true, "16:9": true, "9:16": true, "3:2": true, "2:3": true, "4:3": true, "3:4": true, "auto": true}
imageModels = map[string]bool{"fast": true, "artist": true, "0.2plus": true}
imageJobRe = regexp.MustCompile(`^[A-Za-z0-9_-]{1,64}$`)
imageExt = map[string]string{"image/png": "png", "image/jpeg": "jpg", "image/jpg": "jpg", "image/webp": "webp"}
)
// POST /api/panel/ai/image - {mode: generate|edit, prompt, model, ratio,
// image: {mime, data (base64)} | image_url}: puts the job, answers its id.
func (s *Server) handleAIImage(w http.ResponseWriter, r *http.Request) {
fail := func(msg string) { writeJSON(w, map[string]any{"ok": false, "error": msg}) }
if !s.aiOn(s.currentAdmin(r).ID) {
fail("ассистент выключен в профиле")
return
}
base, key := s.imageConf()
if key == "" {
fail("не задан API-ключ сервиса картинок (Настройки › ИИ-ассистент › Генерация изображений)")
return
}
var in struct {
Mode string `json:"mode"`
Prompt string `json:"prompt"`
Model string `json:"model"`
Ratio string `json:"ratio"`
Enhance *bool `json:"enhance"` // the service improves the description; on unless false
ImageURL string `json:"image_url"`
Image struct {
MIME string `json:"mime"`
Data string `json:"data"`
} `json:"image"`
}
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 32<<20)).Decode(&in); err != nil {
fail("плохой запрос")
return
}
in.Prompt = strings.TrimSpace(in.Prompt)
if in.Prompt == "" {
fail("пустое описание картинки")
return
}
if r := []rune(in.Prompt); len(r) > 4000 {
in.Prompt = string(r[:4000])
}
var job struct {
JobID string `json:"job_id"`
Cost int `json:"cost"`
}
ctx := r.Context()
if in.Mode != "edit" {
if !imageModels[in.Model] {
in.Model = "fast"
}
if !imageRatios[in.Ratio] {
in.Ratio = "1:1"
}
enhance := in.Enhance == nil || *in.Enhance
b, _ := json.Marshal(map[string]any{"prompt": in.Prompt, "model": in.Model, "ratio": in.Ratio, "enhance": enhance})
req, _ := http.NewRequest(http.MethodPost, base+"/api/vertex/v1/generate", bytes.NewReader(b))
req.Header.Set("Content-Type", "application/json")
if err := imageCall(ctx, req, key, &job); err != nil {
fail(err.Error())
return
}
} else {
mime, data, err := s.imageSource(ctx, base, in.Image.MIME, in.Image.Data, in.ImageURL)
if err != nil {
fail(err.Error())
return
}
var body bytes.Buffer
mw := multipart.NewWriter(&body)
h := textproto.MIMEHeader{}
h.Set("Content-Disposition", `form-data; name="image"; filename="photo.`+imageExt[mime]+`"`)
h.Set("Content-Type", mime)
part, _ := mw.CreatePart(h)
_, _ = part.Write(data)
_ = mw.WriteField("prompt", in.Prompt)
_ = mw.Close()
req, _ := http.NewRequest(http.MethodPost, base+"/api/vertex/v1/edit", &body)
req.Header.Set("Content-Type", mw.FormDataContentType())
if err := imageCall(ctx, req, key, &job); err != nil {
fail(err.Error())
return
}
}
if job.JobID == "" {
fail("сервис картинок не дал номер задачи")
return
}
writeJSON(w, map[string]any{"ok": true, "job_id": job.JobID, "cost": job.Cost})
}
// imageSource is the photo to edit: sent by the browser, or a picture made
// before, taken from the service's own address (and from no other).
func (s *Server) imageSource(ctx context.Context, base, mime, b64, picURL string) (string, []byte, error) {
if picURL != "" {
u, err := url.Parse(picURL)
bu, _ := url.Parse(base)
if err != nil || bu == nil || u.Host != bu.Host || (u.Scheme != "https" && u.Scheme != "http") {
return "", nil, errors.New("прежнюю картинку можно править только с сервиса картинок")
}
req, _ := http.NewRequestWithContext(ctx, http.MethodGet, u.String(), nil)
resp, err := imageClient.Do(req)
if err != nil || resp.StatusCode/100 != 2 {
if resp != nil {
resp.Body.Close()
}
return "", nil, errors.New("прежняя картинка больше не открывается")
}
defer resp.Body.Close()
data, _ := io.ReadAll(io.LimitReader(resp.Body, imageMaxBytes+1))
mime = strings.ToLower(strings.SplitN(resp.Header.Get("Content-Type"), ";", 2)[0])
if imageExt[mime] == "" {
mime = "image/png"
}
return mime, data, nil
}
mime = strings.ToLower(mime)
if imageExt[mime] == "" {
return "", nil, errors.New("править можно png, jpg и webp")
}
data, err := base64.StdEncoding.DecodeString(b64)
if err != nil || len(data) == 0 {
return "", nil, errors.New("нет фото для правки: прикрепите его к сообщению")
}
if len(data) > imageMaxBytes {
return "", nil, errors.New("фото больше 20 МБ")
}
return mime, data, nil
}
// GET /api/panel/ai/image/{job}?wait=1..45 - the job's state, waiting up to
// wait seconds for it to end.
func (s *Server) handleAIImageJob(w http.ResponseWriter, r *http.Request) {
fail := func(msg string) { writeJSON(w, map[string]any{"ok": false, "error": msg}) }
if !s.aiOn(s.currentAdmin(r).ID) {
fail("ассистент выключен в профиле")
return
}
base, key := s.imageConf()
id := r.PathValue("job")
if key == "" || !imageJobRe.MatchString(id) {
fail("нет такой задачи")
return
}
wait, _ := strconv.Atoi(r.URL.Query().Get("wait"))
if wait < 0 {
wait = 0
}
if wait > 45 {
wait = 45
}
req, _ := http.NewRequest(http.MethodGet, base+"/api/vertex/v1/jobs/"+id+"?wait="+strconv.Itoa(wait), nil)
var st struct {
Status string `json:"status"`
PhotoURL string `json:"photo_url"`
Cost int `json:"cost"`
Balance int `json:"balance"`
ErrorCode string `json:"error_code"`
Error string `json:"error"`
}
if err := imageCall(r.Context(), req, key, &st); err != nil {
fail(err.Error())
return
}
out := map[string]any{"ok": true, "status": st.Status}
switch st.Status {
case "done":
out["photo_url"], out["cost"], out["balance"] = st.PhotoURL, st.Cost, st.Balance
case "failed":
out["error"] = imageError(st.ErrorCode, st.Error)
}
writeJSON(w, out)
}
+305
View File
@@ -0,0 +1,305 @@
package server
import (
"bytes"
"encoding/base64"
"encoding/json"
"fmt"
"io"
"net/http"
"net/http/httptest"
"net/url"
"strings"
"sync"
"testing"
)
// fakeVertex is the image service: it records what it was asked and answers
// like the real one (a job to poll, a picture when done).
type fakeVertex struct {
srv *httptest.Server
mu sync.Mutex
auth []string
gen map[string]any
edit struct {
prompt string
image []byte
name string
}
polls int
failAs string // an error_code the job ends with
}
func newFakeVertex(t *testing.T) *fakeVertex {
f := &fakeVertex{}
f.srv = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
f.mu.Lock()
defer f.mu.Unlock()
f.auth = append(f.auth, r.Header.Get("Authorization"))
// The pictures are open by their address, as at the real service.
if r.URL.Path == "/uploads/userphoto/ai/t.png" {
w.Header().Set("Content-Type", "image/png")
_, _ = w.Write([]byte("\x89PNG-original"))
return
}
w.Header().Set("Content-Type", "application/json")
if r.Header.Get("Authorization") != "Bearer vx_good" {
w.WriteHeader(401)
fmt.Fprint(w, `{"ok":false,"error_code":"unauthorized","error":"Неверный ключ"}`)
return
}
switch {
case r.URL.Path == "/api/vertex/v1/generate":
_ = json.NewDecoder(r.Body).Decode(&f.gen)
w.WriteHeader(202)
fmt.Fprint(w, `{"ok":true,"job_id":"job_gen1","status":"queued","cost":5}`)
case r.URL.Path == "/api/vertex/v1/edit":
if err := r.ParseMultipartForm(32 << 20); err != nil {
w.WriteHeader(400)
fmt.Fprint(w, `{"ok":false,"error_code":"invalid_image","error":"нет фото"}`)
return
}
f.edit.prompt = r.FormValue("prompt")
file, hdr, err := r.FormFile("image")
if err == nil {
f.edit.image, _ = io.ReadAll(file)
f.edit.name = hdr.Filename
}
w.WriteHeader(202)
fmt.Fprint(w, `{"ok":true,"job_id":"job_edit1","status":"queued","cost":2}`)
case strings.HasPrefix(r.URL.Path, "/api/vertex/v1/jobs/"):
f.polls++
if f.failAs != "" {
fmt.Fprintf(w, `{"ok":true,"job_id":"x","status":"failed","error_code":%q,"error":"сбой"}`, f.failAs)
return
}
if f.polls < 2 {
fmt.Fprint(w, `{"ok":true,"job_id":"x","status":"running"}`)
return
}
fmt.Fprintf(w, `{"ok":true,"job_id":"x","status":"done","photo_url":"%s/uploads/userphoto/ai/t.png","cost":5,"balance":42}`, f.srv.URL)
default:
http.NotFound(w, r)
}
}))
t.Cleanup(f.srv.Close)
return f
}
// setImage points the panel at the fake service with a key.
func setImage(t *testing.T, s *Server, f *fakeVertex, key string) {
t.Helper()
_ = s.db.SetConfig(cfgImageURL, f.srv.URL)
_ = s.db.SetConfig(cfgImageKey, key)
}
func apiCall(t *testing.T, s *Server, c *http.Cookie, method, path, body string) string {
t.Helper()
req := httptest.NewRequest(method, path, strings.NewReader(body))
req.AddCookie(c)
w := httptest.NewRecorder()
s.Handler().ServeHTTP(w, req)
return w.Body.String()
}
// A picture from nothing: the prompt, model and ratio go to the service with
// the key; the job's progress and its picture come back, the key never.
func TestImageGenerate(t *testing.T) {
s, c := newTestServer(t)
f := newFakeVertex(t)
setImage(t, s, f, "vx_good")
got := apiCall(t, s, c, "POST", "/api/panel/ai/image", `{"mode":"generate","prompt":"cat in space","model":"artist","ratio":"3:4"}`)
if !strings.Contains(got, `"ok":true`) || !strings.Contains(got, `"job_id":"job_gen1"`) || strings.Contains(got, "vx_good") {
t.Fatalf("create: %s", got)
}
if f.gen["prompt"] != "cat in space" || f.gen["model"] != "artist" || f.gen["ratio"] != "3:4" || f.gen["enhance"] != true {
t.Fatalf("what the service got: %v", f.gen)
}
if got := apiCall(t, s, c, "GET", "/api/panel/ai/image/job_gen1?wait=1", ""); !strings.Contains(got, `"status":"running"`) {
t.Fatalf("first poll: %s", got)
}
got = apiCall(t, s, c, "GET", "/api/panel/ai/image/job_gen1?wait=1", "")
if !strings.Contains(got, `"status":"done"`) || !strings.Contains(got, "/uploads/userphoto/ai/t.png") || !strings.Contains(got, `"balance":42`) || strings.Contains(got, "vx_good") {
t.Fatalf("done: %s", got)
}
}
// Unknown models and ratios are not sent on: the defaults go instead.
func TestImageGenerateDefaults(t *testing.T) {
s, c := newTestServer(t)
f := newFakeVertex(t)
setImage(t, s, f, "vx_good")
apiCall(t, s, c, "POST", "/api/panel/ai/image", `{"mode":"generate","prompt":"x","model":"gpt","ratio":"7:1"}`)
if f.gen["model"] != "fast" || f.gen["ratio"] != "1:1" {
t.Fatalf("defaults: %v", f.gen)
}
}
// Editing sends the user's photo as a file with the instruction.
func TestImageEditFromAttachedPhoto(t *testing.T) {
s, c := newTestServer(t)
f := newFakeVertex(t)
setImage(t, s, f, "vx_good")
photo := []byte("\xff\xd8\xffJPEGDATA")
b, _ := json.Marshal(map[string]any{"mode": "edit", "prompt": "сделай закат",
"image": map[string]string{"mime": "image/jpeg", "data": base64.StdEncoding.EncodeToString(photo)}})
got := apiCall(t, s, c, "POST", "/api/panel/ai/image", string(b))
if !strings.Contains(got, `"job_id":"job_edit1"`) {
t.Fatalf("edit: %s", got)
}
if f.edit.prompt != "сделай закат" || !bytes.Equal(f.edit.image, photo) || !strings.HasSuffix(f.edit.name, ".jpg") {
t.Fatalf("what the service got: %q %q %q", f.edit.prompt, f.edit.image, f.edit.name)
}
// A format the service does not take is told so, not sent.
b, _ = json.Marshal(map[string]any{"mode": "edit", "prompt": "x", "image": map[string]string{"mime": "image/gif", "data": "R0lG"}})
if got := apiCall(t, s, c, "POST", "/api/panel/ai/image", string(b)); !strings.Contains(got, `"ok":false`) || !strings.Contains(got, "png") {
t.Fatalf("gif: %s", got)
}
}
// A picture made before can be edited again: fetched from the service's own
// address only - never from another.
func TestImageEditFromEarlierPicture(t *testing.T) {
s, c := newTestServer(t)
f := newFakeVertex(t)
setImage(t, s, f, "vx_good")
b, _ := json.Marshal(map[string]any{"mode": "edit", "prompt": "темнее", "image_url": f.srv.URL + "/uploads/userphoto/ai/t.png"})
if got := apiCall(t, s, c, "POST", "/api/panel/ai/image", string(b)); !strings.Contains(got, `"ok":true`) {
t.Fatalf("by url: %s", got)
}
if !bytes.Equal(f.edit.image, []byte("\x89PNG-original")) || !strings.HasSuffix(f.edit.name, ".png") {
t.Fatalf("the picture sent: %q %q", f.edit.image, f.edit.name)
}
b, _ = json.Marshal(map[string]any{"mode": "edit", "prompt": "x", "image_url": "http://127.0.0.1:9/secret.png"})
if got := apiCall(t, s, c, "POST", "/api/panel/ai/image", string(b)); !strings.Contains(got, `"ok":false`) {
t.Fatalf("a foreign address was fetched: %s", got)
}
}
// What goes wrong is told in words: no key, a wrong key, the service's own
// codes, a job that failed.
func TestImageErrors(t *testing.T) {
s, c := newTestServer(t)
f := newFakeVertex(t)
if got := apiCall(t, s, c, "POST", "/api/panel/ai/image", `{"mode":"generate","prompt":"x"}`); !strings.Contains(got, "ключ") {
t.Fatalf("no key: %s", got)
}
setImage(t, s, f, "vx_bad")
if got := apiCall(t, s, c, "POST", "/api/panel/ai/image", `{"mode":"generate","prompt":"x"}`); !strings.Contains(got, `"ok":false`) || !strings.Contains(got, "ключ") {
t.Fatalf("wrong key: %s", got)
}
setImage(t, s, f, "vx_good")
f.failAs = "insufficient_credits"
if got := apiCall(t, s, c, "GET", "/api/panel/ai/image/job_gen1", ""); !strings.Contains(got, `"status":"failed"`) || !strings.Contains(got, "кредит") {
t.Fatalf("failed job: %s", got)
}
if got := apiCall(t, s, c, "GET", "/api/panel/ai/image/..%2Fme", ""); !strings.Contains(got, `"ok":false`) {
t.Fatalf("a job id out of shape: %s", got)
}
if got := apiCall(t, s, c, "POST", "/api/panel/ai/image", `{"mode":"generate","prompt":""}`); !strings.Contains(got, `"ok":false`) {
t.Fatalf("empty prompt: %s", got)
}
}
// The assistant off in the profile means no pictures either.
func TestImageNeedsAssistantOn(t *testing.T) {
s, c := newTestServer(t)
f := newFakeVertex(t)
setImage(t, s, f, "vx_good")
a, _ := s.db.AdminByUsername("adm")
_ = s.db.SetConfig(aiUserKey(a.ID), "0")
if got := apiCall(t, s, c, "POST", "/api/panel/ai/image", `{"mode":"generate","prompt":"x"}`); !strings.Contains(got, "выключен") {
t.Fatalf("off: %s", got)
}
}
// The key is saved from Настройки, shown back to the admin, and the chat
// learns only that pictures are on.
func TestImageKeySetting(t *testing.T) {
s, c := newTestServer(t)
postAIModels(t, s, c, [][7]string{{"", "M", "gradio", "", "", "", ""}})
f := url.Values{"csrf": {"tok"}, "m_type": {"gradio"}, "ai_image_key": {"vx_secret"}}
req := httptest.NewRequest(http.MethodPost, "/settings/ai", strings.NewReader(f.Encode()))
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
req.AddCookie(c)
req.AddCookie(&http.Cookie{Name: csrfCookie, Value: "tok"})
s.Handler().ServeHTTP(httptest.NewRecorder(), req)
if _, k := s.imageConf(); k != "vx_secret" {
t.Fatalf("key not saved: %q", k)
}
if body := get(t, s, c, "/settings?frag=1").Body.String(); !strings.Contains(body, `value="vx_secret"`) {
t.Fatal("the settings page does not show the key")
}
state := get(t, s, c, "/api/panel/ai").Body.String()
if strings.Contains(state, "vx_secret") || !strings.Contains(state, `"image":true`) {
t.Fatalf("state: %s", state)
}
}
// With a key the model is told it can draw; without, it is not.
func TestImagePromptOnlyWithKey(t *testing.T) {
s, _ := newTestServer(t)
if strings.Contains(s.imagePrompt(), "```image") {
t.Fatal("told about pictures with no key")
}
_ = s.db.SetConfig(cfgImageKey, "vx_good")
p := s.imagePrompt()
for _, want := range []string{"```image\n", "```image edit", "fast", "artist", "0.2plus", "16:9"} {
if !strings.Contains(p, want) {
t.Fatalf("the prompt lacks %q:\n%s", want, p)
}
}
}
// The chat gives the model the image instructions when pictures are on.
func TestAIChatMentionsImageTool(t *testing.T) {
s, c := newTestServer(t)
var system string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
var b map[string]any
_ = json.NewDecoder(r.Body).Decode(&b)
system, _ = b["messages"].([]any)[0].(map[string]any)["content"].(string)
fmt.Fprint(w, openAIChunk("content", "ok")+"data: [DONE]\n\n")
}))
defer srv.Close()
postAIModels(t, s, c, [][7]string{{"", "M", "openai", srv.URL, "m", "", ""}})
ask := func() {
apiCall(t, s, c, "POST", "/api/panel/ai/chat", `{"message":"нарисуй кота","context":"x"}`)
}
ask()
if strings.Contains(system, "```image") {
t.Fatal("image tool offered with no key")
}
_ = s.db.SetConfig(cfgImageKey, "vx_good")
ask()
if !strings.Contains(system, "```image") {
t.Fatalf("image tool not offered with a key:\n%s", system)
}
}
// The description is improved by the service unless the model asks for it
// as it is.
func TestImageEnhanceSwitch(t *testing.T) {
s, c := newTestServer(t)
f := newFakeVertex(t)
setImage(t, s, f, "vx_good")
apiCall(t, s, c, "POST", "/api/panel/ai/image", `{"mode":"generate","prompt":"x"}`)
if f.gen["enhance"] != true {
t.Fatalf("by default: %v", f.gen)
}
apiCall(t, s, c, "POST", "/api/panel/ai/image", `{"mode":"generate","prompt":"x","enhance":false}`)
if f.gen["enhance"] != false {
t.Fatalf("switched off: %v", f.gen)
}
apiCall(t, s, c, "POST", "/api/panel/ai/image", `{"mode":"generate","prompt":"x","enhance":true}`)
if f.gen["enhance"] != true {
t.Fatalf("switched on: %v", f.gen)
}
{
p := s.imagePrompt()
_ = s.db.SetConfig(cfgImageKey, "vx_good")
if !strings.Contains(s.imagePrompt(), "noenhance") {
t.Fatalf("the model is not told of the switch:\n%s", p)
}
}
}
+644
View File
@@ -0,0 +1,644 @@
package server
import (
"bytes"
"crypto"
"crypto/ecdsa"
"crypto/ed25519"
"crypto/elliptic"
"crypto/rand"
"crypto/rsa"
"crypto/sha256"
"crypto/x509"
"encoding/base64"
"encoding/binary"
"encoding/json"
"errors"
"fmt"
"math/big"
"net"
"net/http"
"net/url"
"strconv"
"strings"
"sync"
"time"
)
// Passkeys (WebAuthn): an admin signs in with a key held by the device - a
// fingerprint, the phone's lock, a hardware key - instead of a password. With
// a key in place, the admin can turn the password off in their account.
//
// Attestation is not checked ("none"): the panel trusts any authenticator,
// it only needs the public key and, at sign-in, a signature made with it.
var b64u = base64.RawURLEncoding
// challenges are the ones handed out and not yet used, a few minutes each.
type challenges struct {
mu sync.Mutex
m map[string]challenge
}
type challenge struct {
adminID int64 // the admin registering a key; 0 for a sign-in
expires time.Time
}
func (c *challenges) issue(adminID int64) string {
b := make([]byte, 32)
if _, err := rand.Read(b); err != nil {
panic("crypto/rand failed: " + err.Error())
}
id := b64u.EncodeToString(b)
c.mu.Lock()
defer c.mu.Unlock()
if c.m == nil {
c.m = map[string]challenge{}
}
for k, v := range c.m {
if time.Now().After(v.expires) {
delete(c.m, k)
}
}
c.m[id] = challenge{adminID: adminID, expires: time.Now().Add(5 * time.Minute)}
return id
}
// take uses up a challenge: true when it was issued, for this admin, and is
// still fresh.
func (c *challenges) take(id string, adminID int64) bool {
c.mu.Lock()
defer c.mu.Unlock()
ch, ok := c.m[id]
delete(c.m, id)
return ok && ch.adminID == adminID && time.Now().Before(ch.expires)
}
// mainHost is the main address's host name, set when the server starts.
var mainHost = func() string { return "" }
// rpID is the site the keys belong to: the host the browser is on, or the
// main address's host when the browser is on it or under it - so the keys
// made on lpra.example.com also sign in on direct.lpra.example.com.
func rpID(r *http.Request) string {
host := r.Host
if h, _, err := net.SplitHostPort(host); err == nil {
host = h
}
if m := mainHost(); m != "" && strings.HasSuffix(host, "."+m) {
return m
}
return host
}
// checkClientData verifies what the browser signed over: the kind of
// ceremony, our challenge and our origin. A proxy in front may speak plain
// http to us, so the origin is matched by host, and must be https unless the
// panel is on this machine.
func checkClientData(r *http.Request, raw []byte, typ string, adminID int64, ch *challenges) error {
var cd struct {
Type string `json:"type"`
Challenge string `json:"challenge"`
Origin string `json:"origin"`
}
if err := json.Unmarshal(raw, &cd); err != nil {
return errors.New("ответ ключа не читается")
}
if cd.Type != typ {
return errors.New("ответ ключа не того вида")
}
if !ch.take(cd.Challenge, adminID) {
return errors.New("запрос устарел - попробуйте ещё раз")
}
u, err := url.Parse(cd.Origin)
if err != nil || u.Host != r.Host {
return errors.New("ключ подписал запрос для другого сайта")
}
local := u.Hostname() == "localhost" || u.Hostname() == "127.0.0.1" || u.Hostname() == "::1"
if u.Scheme != "https" && !local {
return errors.New("ключи доступа работают только по https")
}
return nil
}
// authData is the authenticator's part: rpIdHash, flags, counter, then the
// new credential at registration.
type authData struct {
rpIDHash []byte
flags byte
signCount uint32
credID []byte
coseKey []byte
}
func parseAuthData(b []byte, wantCred bool) (*authData, error) {
if len(b) < 37 {
return nil, errors.New("данные ключа слишком короткие")
}
ad := &authData{rpIDHash: b[:32], flags: b[32], signCount: binary.BigEndian.Uint32(b[33:37])}
if !wantCred {
return ad, nil
}
if ad.flags&0x40 == 0 || len(b) < 37+18 {
return nil, errors.New("ключ не прислал себя")
}
n := int(binary.BigEndian.Uint16(b[53:55]))
if len(b) < 55+n {
return nil, errors.New("данные ключа обрезаны")
}
ad.credID = b[55 : 55+n]
ad.coseKey = b[55+n:]
return ad, nil
}
func (ad *authData) check(r *http.Request) error {
sum := sha256.Sum256([]byte(rpID(r)))
if !bytes.Equal(ad.rpIDHash, sum[:]) {
return errors.New("ключ создан для другого сайта")
}
if ad.flags&0x01 == 0 {
return errors.New("ключ не подтвердил присутствие")
}
return nil
}
// coseToPKIX turns a COSE public key (ES256, RS256 or EdDSA) into PKIX DER.
func coseToPKIX(raw []byte) ([]byte, error) {
v, _, err := cborDecode(raw)
if err != nil {
return nil, err
}
m, ok := v.(map[any]any)
if !ok {
return nil, errors.New("ключ не в формате COSE")
}
num := func(k int64) int64 { n, _ := m[k].(int64); return n }
bin := func(k int64) []byte { b, _ := m[k].([]byte); return b }
var pub any
switch num(1) { // kty
case 2: // EC2
x, y := bin(-2), bin(-3)
if num(-1) != 1 || len(x) != 32 || len(y) != 32 {
return nil, errors.New("поддерживается только кривая P-256")
}
k := &ecdsa.PublicKey{Curve: elliptic.P256(), X: new(big.Int).SetBytes(x), Y: new(big.Int).SetBytes(y)}
if !k.Curve.IsOnCurve(k.X, k.Y) {
return nil, errors.New("ключ не на кривой")
}
pub = k
case 3: // RSA
n, e := bin(-1), bin(-2)
if len(n) < 256 || len(e) == 0 || len(e) > 4 {
return nil, errors.New("неподходящий ключ RSA")
}
pub = &rsa.PublicKey{N: new(big.Int).SetBytes(n), E: int(new(big.Int).SetBytes(e).Int64())}
case 1: // OKP
x := bin(-2)
if num(-1) != 6 || len(x) != ed25519.PublicKeySize {
return nil, errors.New("поддерживается только Ed25519")
}
pub = ed25519.PublicKey(x)
default:
return nil, errors.New("неизвестный вид ключа")
}
return x509.MarshalPKIXPublicKey(pub)
}
func verifySig(der, data, sig []byte) bool {
pub, err := x509.ParsePKIXPublicKey(der)
if err != nil {
return false
}
sum := sha256.Sum256(data)
switch k := pub.(type) {
case *ecdsa.PublicKey:
return ecdsa.VerifyASN1(k, sum[:], sig)
case *rsa.PublicKey:
return rsa.VerifyPKCS1v15(k, crypto.SHA256, sum[:], sig) == nil
case ed25519.PublicKey:
return ed25519.Verify(k, data, sig)
}
return false
}
// ---------- CBOR, as much as WebAuthn needs ----------
func cborDecode(b []byte) (any, []byte, error) {
r := &cborReader{b: b}
v, err := r.item(0)
return v, r.b[r.i:], err
}
type cborReader struct {
b []byte
i int
}
var errCBOR = errors.New("данные ключа повреждены")
func (r *cborReader) arg(info byte) (uint64, error) {
size := map[byte]int{24: 1, 25: 2, 26: 4, 27: 8}
if info < 24 {
return uint64(info), nil
}
n, ok := size[info]
if !ok || r.i+n > len(r.b) {
return 0, errCBOR
}
var v uint64
for _, c := range r.b[r.i : r.i+n] {
v = v<<8 | uint64(c)
}
r.i += n
return v, nil
}
func (r *cborReader) take(n uint64) ([]byte, error) {
if n > uint64(len(r.b)-r.i) {
return nil, errCBOR
}
out := r.b[r.i : r.i+int(n)]
r.i += int(n)
return out, nil
}
func (r *cborReader) item(depth int) (any, error) {
if r.i >= len(r.b) || depth > 16 {
return nil, errCBOR
}
ib := r.b[r.i]
r.i++
major, info := ib>>5, ib&31
if major == 7 {
switch info {
case 20:
return false, nil
case 21:
return true, nil
case 22, 23:
return nil, nil
}
_, err := r.arg(info) // a float: not used, skipped
return nil, err
}
n, err := r.arg(info)
if err != nil {
return nil, err
}
switch major {
case 0:
return int64(n), nil
case 1:
return -1 - int64(n), nil
case 2:
return r.take(n)
case 3:
s, err := r.take(n)
return string(s), err
case 4:
if n > 1024 {
return nil, errCBOR
}
arr := make([]any, 0, n)
for k := uint64(0); k < n; k++ {
v, err := r.item(depth + 1)
if err != nil {
return nil, err
}
arr = append(arr, v)
}
return arr, nil
case 5:
if n > 1024 {
return nil, errCBOR
}
m := make(map[any]any, n)
for k := uint64(0); k < n; k++ {
key, err := r.item(depth + 1)
if err != nil {
return nil, err
}
v, err := r.item(depth + 1)
if err != nil {
return nil, err
}
switch key.(type) {
case int64, string:
m[key] = v
default:
return nil, errCBOR
}
}
return m, nil
case 6:
return r.item(depth + 1) // a tag: its content
}
return nil, errCBOR
}
// ---------- handlers ----------
func credParams() []map[string]any {
return []map[string]any{
{"type": "public-key", "alg": -7}, // ES256
{"type": "public-key", "alg": -8}, // EdDSA
{"type": "public-key", "alg": -257}, // RS256
}
}
func userHandle(id int64) string {
b := make([]byte, 8)
binary.BigEndian.PutUint64(b, uint64(id))
return b64u.EncodeToString(b)
}
// GET /api/panel/account - who is signed in, and their passkeys.
func (s *Server) handleAccount(w http.ResponseWriter, r *http.Request) {
a := s.currentAdmin(r)
keys, _ := s.db.ListAdminKeys(a.ID)
list := []map[string]any{}
for _, k := range keys {
list = append(list, map[string]any{"id": k.ID, "name": k.Name, "created": k.CreatedAt, "last_used": k.LastUsed})
}
writeJSON(w, map[string]any{"username": a.Username, "keys": list, "password_off": s.db.PasswordOff(a.ID), "pin": s.db.AdminPin(a.ID) != "",
"pw_locked": a.PwLocked, "admin": a.IsAdmin()})
}
// POST /api/panel/account/password/change - a new password, given the
// current one.
func (s *Server) handlePasswordChange(w http.ResponseWriter, r *http.Request) {
a := s.currentAdmin(r)
var in struct {
Current string `json:"current"`
New string `json:"new"`
}
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 4<<10)).Decode(&in); err != nil {
writeJSON(w, map[string]any{"ok": false, "error": "плохой запрос"})
return
}
ip := clientIP(r)
if s.limiter.blocked(ip) {
writeJSON(w, map[string]any{"ok": false, "error": "Слишком много попыток. Подождите немного."})
return
}
if !checkPassword(a.PassHash, in.Current) {
s.limiter.fin(ip, false)
writeJSON(w, map[string]any{"ok": false, "error": "текущий пароль не тот"})
return
}
if len(in.New) < 8 {
writeJSON(w, map[string]any{"ok": false, "error": "новый пароль - не короче 8 символов"})
return
}
hash, err := hashPassword(in.New)
if err == nil {
err = s.db.SetAdminPassword(a.ID, hash)
}
if err != nil {
writeJSON(w, map[string]any{"ok": false, "error": err.Error()})
return
}
s.limiter.fin(ip, true)
s.db.Audit(a.ID, "", "password_change")
writeJSON(w, map[string]any{"ok": true})
}
// POST /api/panel/account/password {"off": bool} - turn signing in with the
// password off or on again. Off only with a passkey to sign in with.
func (s *Server) handlePasswordToggle(w http.ResponseWriter, r *http.Request) {
a := s.currentAdmin(r)
var in struct {
Off bool `json:"off"`
}
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 1<<10)).Decode(&in); err != nil {
writeJSON(w, map[string]any{"ok": false, "error": "плохой запрос"})
return
}
if a.PwLocked {
writeJSON(w, map[string]any{"ok": false, "error": "вход по паролю закрепил администратор"})
return
}
if in.Off && s.db.CountAdminKeys(a.ID) == 0 {
writeJSON(w, map[string]any{"ok": false, "error": "сначала добавьте ключ доступа"})
return
}
if err := s.db.SetPasswordOff(a.ID, in.Off); err != nil {
writeJSON(w, map[string]any{"ok": false, "error": err.Error()})
return
}
action := "password_on"
if in.Off {
action = "password_off"
}
s.db.Audit(a.ID, "", action)
writeJSON(w, map[string]any{"ok": true})
}
// POST /api/panel/account/keys/begin - options to make a new passkey with.
func (s *Server) handleKeyRegisterBegin(w http.ResponseWriter, r *http.Request) {
a := s.currentAdmin(r)
keys, _ := s.db.ListAdminKeys(a.ID)
exclude := []map[string]any{}
for _, k := range keys {
exclude = append(exclude, map[string]any{"type": "public-key", "id": b64u.EncodeToString(k.CredID)})
}
writeJSON(w, map[string]any{"ok": true, "options": map[string]any{
"challenge": s.challenges.issue(a.ID),
"rp": map[string]any{"id": rpID(r), "name": "LPRA"},
"user": map[string]any{"id": userHandle(a.ID), "name": a.Username, "displayName": a.Username},
"pubKeyCredParams": credParams(),
"excludeCredentials": exclude,
"authenticatorSelection": map[string]any{
"residentKey": "preferred", "userVerification": "preferred",
},
"attestation": "none",
"timeout": 120000,
}})
}
// POST /api/panel/account/keys/finish - keep the passkey the browser made.
func (s *Server) handleKeyRegisterFinish(w http.ResponseWriter, r *http.Request) {
a := s.currentAdmin(r)
var in struct {
Name string `json:"name"`
ClientDataJSON string `json:"clientDataJSON"`
AttestationObject string `json:"attestationObject"`
}
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 64<<10)).Decode(&in); err != nil {
writeJSON(w, map[string]any{"ok": false, "error": "плохой запрос"})
return
}
fail := func(err error) { writeJSON(w, map[string]any{"ok": false, "error": err.Error()}) }
cd, err1 := b64u.DecodeString(in.ClientDataJSON)
att, err2 := b64u.DecodeString(in.AttestationObject)
if err1 != nil || err2 != nil {
fail(errors.New("плохой запрос"))
return
}
if err := checkClientData(r, cd, "webauthn.create", a.ID, &s.challenges); err != nil {
fail(err)
return
}
v, _, err := cborDecode(att)
m, ok := v.(map[any]any)
if err != nil || !ok {
fail(errCBOR)
return
}
raw, _ := m["authData"].([]byte)
ad, err := parseAuthData(raw, true)
if err != nil {
fail(err)
return
}
if err := ad.check(r); err != nil {
fail(err)
return
}
der, err := coseToPKIX(ad.coseKey)
if err != nil {
fail(err)
return
}
if k, _ := s.db.AdminKeyByCredID(ad.credID); k != nil {
fail(errors.New("этот ключ уже добавлен"))
return
}
name := strings.TrimSpace(in.Name)
if name == "" {
name = "ключ " + time.Now().Format("02.01.2006")
}
if len(name) > 60 {
name = name[:60]
}
if err := s.db.AddAdminKey(AdminKey{AdminID: a.ID, CredID: ad.credID, PublicKey: der, SignCount: ad.signCount, Name: name}); err != nil {
fail(err)
return
}
s.db.Audit(a.ID, "", "passkey_add:"+name)
writeJSON(w, map[string]any{"ok": true})
}
// POST /api/panel/account/keys/{id}/delete
func (s *Server) handleKeyDelete(w http.ResponseWriter, r *http.Request) {
a := s.currentAdmin(r)
id, _ := strconv.ParseInt(r.PathValue("id"), 10, 64)
if err := s.db.DeleteAdminKey(a.ID, id); err != nil {
writeJSON(w, map[string]any{"ok": false, "error": err.Error()})
return
}
s.db.Audit(a.ID, "", "passkey_delete:"+strconv.FormatInt(id, 10))
writeJSON(w, map[string]any{"ok": true})
}
// POST /api/login/key/begin - options to sign in with a passkey. With a
// username, its keys are named, for keys that do not keep the account
// themselves.
func (s *Server) handleKeyLoginBegin(w http.ResponseWriter, r *http.Request) {
if s.limiter.blocked(clientIP(r)) {
writeJSON(w, map[string]any{"ok": false, "error": "Слишком много попыток. Подождите немного."})
return
}
var in struct {
Username string `json:"username"`
}
_ = json.NewDecoder(http.MaxBytesReader(w, r.Body, 4<<10)).Decode(&in)
allow := []map[string]any{}
if u := strings.TrimSpace(in.Username); u != "" {
if a, _ := s.db.AdminByUsername(u); a != nil {
keys, _ := s.db.ListAdminKeys(a.ID)
for _, k := range keys {
allow = append(allow, map[string]any{"type": "public-key", "id": b64u.EncodeToString(k.CredID)})
}
}
}
writeJSON(w, map[string]any{"ok": true, "options": map[string]any{
"challenge": s.challenges.issue(0),
"rpId": rpID(r),
"allowCredentials": allow,
"userVerification": "preferred",
"timeout": 120000,
}})
}
// POST /api/login/key/finish - check the signature and sign the admin in.
func (s *Server) handleKeyLoginFinish(w http.ResponseWriter, r *http.Request) {
ip := clientIP(r)
if s.limiter.blocked(ip) {
writeJSON(w, map[string]any{"ok": false, "error": "Слишком много попыток. Подождите немного."})
return
}
var in struct {
ID string `json:"id"`
ClientDataJSON string `json:"clientDataJSON"`
AuthenticatorData string `json:"authenticatorData"`
Signature string `json:"signature"`
}
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 64<<10)).Decode(&in); err != nil {
writeJSON(w, map[string]any{"ok": false, "error": "плохой запрос"})
return
}
fail := func(msg string) {
s.limiter.fin(ip, false)
writeJSON(w, map[string]any{"ok": false, "error": msg})
}
credID, e1 := b64u.DecodeString(in.ID)
cd, e2 := b64u.DecodeString(in.ClientDataJSON)
rawAD, e3 := b64u.DecodeString(in.AuthenticatorData)
sig, e4 := b64u.DecodeString(in.Signature)
if e1 != nil || e2 != nil || e3 != nil || e4 != nil {
fail("плохой запрос")
return
}
k, _ := s.db.AdminKeyByCredID(credID)
if k == nil {
fail("этот ключ здесь не привязан ни к одному аккаунту")
return
}
if err := checkClientData(r, cd, "webauthn.get", 0, &s.challenges); err != nil {
fail(err.Error())
return
}
ad, err := parseAuthData(rawAD, false)
if err == nil {
err = ad.check(r)
}
if err != nil {
fail(err.Error())
return
}
cdHash := sha256.Sum256(cd)
if !verifySig(k.PublicKey, append(append([]byte{}, rawAD...), cdHash[:]...), sig) {
fail("подпись ключа не сошлась")
return
}
// A counter that goes back means a copy of the key is in use.
if ad.signCount != 0 && k.SignCount != 0 && ad.signCount <= k.SignCount {
fail("ключ похож на копию: счётчик не вырос")
return
}
_ = s.db.UseAdminKey(k.ID, ad.signCount)
s.limiter.fin(ip, true)
if err := s.setSession(w, k.AdminID); err != nil {
writeJSON(w, map[string]any{"ok": false, "error": "внутренняя ошибка"})
return
}
s.db.Audit(k.AdminID, "", "login_passkey:"+k.Name)
writeJSON(w, map[string]any{"ok": true})
}
// POST /settings/admins/keys/reset - drop all passkeys of an admin, who signs
// in with the password again: for one who lost theirs.
func (s *Server) handleResetAdminKeys(w http.ResponseWriter, r *http.Request) {
if !checkCSRF(r) {
http.Error(w, "bad csrf", http.StatusForbidden)
return
}
id, _ := strconv.ParseInt(r.FormValue("id"), 10, 64)
_ = s.db.DeleteAdminKeys(id)
if a := s.currentAdmin(r); a != nil {
s.db.Audit(a.ID, "", fmt.Sprintf("passkeys_reset:%d", id))
}
http.Redirect(w, r, "/settings", http.StatusSeeOther)
}
+108
View File
@@ -0,0 +1,108 @@
package server
import (
"encoding/json"
"net/http"
"regexp"
"sync"
)
// The PIN: four digits an admin can set, asked for by the page in every new
// browser tab (a reload of the same tab does not ask again). The server
// checks it; five wrong ones in a row end the session.
var pinRe = regexp.MustCompile(`^[0-9]{4}$`)
// pinMisses counts wrong PINs per session.
type pinMisses struct {
mu sync.Mutex
m map[string]int
}
func (p *pinMisses) miss(sid string) int {
p.mu.Lock()
defer p.mu.Unlock()
if p.m == nil {
p.m = map[string]int{}
}
p.m[sid]++
return p.m[sid]
}
func (p *pinMisses) clear(sid string) {
p.mu.Lock()
delete(p.m, sid)
p.mu.Unlock()
}
// GET /api/panel/pin - whether the admin has a PIN to ask for.
func (s *Server) handlePinState(w http.ResponseWriter, r *http.Request) {
a := s.currentAdmin(r)
w.Header().Set("Cache-Control", "no-store")
writeJSON(w, map[string]any{"pin": s.db.AdminPin(a.ID) != ""})
}
// POST /api/panel/pin/check {"pin"} - the PIN typed in a new tab.
func (s *Server) handlePinCheck(w http.ResponseWriter, r *http.Request) {
a := s.currentAdmin(r)
var in struct {
Pin string `json:"pin"`
}
_ = json.NewDecoder(http.MaxBytesReader(w, r.Body, 1<<10)).Decode(&in)
h := s.db.AdminPin(a.ID)
c, _ := r.Cookie(sessionCookie)
if h == "" || (pinRe.MatchString(in.Pin) && checkPassword(h, in.Pin)) {
s.pinMisses.clear(c.Value)
writeJSON(w, map[string]any{"ok": true})
return
}
if n := s.pinMisses.miss(c.Value); n >= 5 {
s.pinMisses.clear(c.Value)
s.clearSession(w, r)
s.db.Audit(a.ID, "", "pin_lockout")
writeJSON(w, map[string]any{"ok": false, "out": true, "error": "слишком много неверных попыток"})
return
} else {
writeJSON(w, map[string]any{"ok": false, "left": 5 - n, "error": "неверный PIN"})
}
}
// POST /api/panel/account/pin {"pin", "current"} - set or change the PIN;
// with pin empty, remove it. A PIN in place is needed to change or drop it.
func (s *Server) handlePinSet(w http.ResponseWriter, r *http.Request) {
a := s.currentAdmin(r)
var in struct {
Pin string `json:"pin"`
Current string `json:"current"`
}
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 1<<10)).Decode(&in); err != nil {
writeJSON(w, map[string]any{"ok": false, "error": "плохой запрос"})
return
}
if h := s.db.AdminPin(a.ID); h != "" && !checkPassword(h, in.Current) {
writeJSON(w, map[string]any{"ok": false, "error": "текущий PIN не тот"})
return
}
hash := ""
if in.Pin != "" {
if !pinRe.MatchString(in.Pin) {
writeJSON(w, map[string]any{"ok": false, "error": "PIN - ровно 4 цифры"})
return
}
var err error
if hash, err = hashPassword(in.Pin); err != nil {
writeJSON(w, map[string]any{"ok": false, "error": err.Error()})
return
}
}
if err := s.db.SetAdminPin(a.ID, hash); err != nil {
writeJSON(w, map[string]any{"ok": false, "error": err.Error()})
return
}
action := "pin_set"
if hash == "" {
action = "pin_remove"
}
s.db.Audit(a.ID, "", action)
writeJSON(w, map[string]any{"ok": true})
}
+80
View File
@@ -0,0 +1,80 @@
package server
import (
"encoding/json"
"net/http"
"strconv"
"strings"
)
// Each user's look of the panel (Аккаунт › Вид), kept on the server so it
// is the same on every device: where a tab's icons go, the terminal's font
// size, a compact file list, the time column, animations, and whether a
// terminal or file pane opened «Здесь» starts in the active pane's folder;
// and the assistant's model: which one, and whether the chat always offers
// the choice.
type viewPrefs struct {
TabIcons string `json:"tabIcons,omitempty"` // left (default), right, hide
TermFont int `json:"termFont,omitempty"` // 11-20; 0: the default
Compact bool `json:"compact,omitempty"` // denser file lists
FileTime string `json:"fileTime,omitempty"` // show (default), hide
Motion string `json:"motion,omitempty"` // on (default), off
OpenHere string `json:"openHere,omitempty"` // on (default), off
// AIModelPick: the model menu in the chat - when there are several
// (default) or always. AIModel: the id of the model chosen.
AIModelPick string `json:"aiModelPick,omitempty"`
AIModel string `json:"aiModel,omitempty"`
}
func prefsKey(id int64) string { return "prefs_" + strconv.FormatInt(id, 10) }
// clean keeps known values only.
func (p viewPrefs) clean() viewPrefs {
if p.TabIcons != "right" && p.TabIcons != "hide" {
p.TabIcons = ""
}
if p.TermFont < 11 || p.TermFont > 20 {
p.TermFont = 0
}
if p.FileTime != "hide" {
p.FileTime = ""
}
if p.Motion != "off" {
p.Motion = ""
}
if p.OpenHere != "off" {
p.OpenHere = ""
}
if p.AIModelPick != "always" {
p.AIModelPick = ""
}
if len(p.AIModel) > 32 || strings.Trim(p.AIModel, "0123456789abcdefghijklmnopqrstuvwxyz") != "" {
p.AIModel = ""
}
return p
}
func (s *Server) prefs(id int64) viewPrefs {
var p viewPrefs
if v, _ := s.db.GetConfig(prefsKey(id)); v != "" {
_ = json.Unmarshal([]byte(v), &p)
}
return p.clean()
}
// GET /api/panel/account/prefs
func (s *Server) handlePrefs(w http.ResponseWriter, r *http.Request) {
writeJSON(w, s.prefs(s.currentAdmin(r).ID))
}
// POST /api/panel/account/prefs - the whole set.
func (s *Server) handlePrefsSet(w http.ResponseWriter, r *http.Request) {
var p viewPrefs
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 4<<10)).Decode(&p); err != nil {
writeJSON(w, map[string]any{"ok": false, "error": "плохой запрос"})
return
}
b, _ := json.Marshal(p.clean())
_ = s.db.SetConfig(prefsKey(s.currentAdmin(r).ID), string(b))
writeJSON(w, map[string]any{"ok": true})
}
+282
View File
@@ -0,0 +1,282 @@
package server
import (
"context"
"encoding/json"
"errors"
"html"
"io"
"log"
"net/http"
"net/url"
"regexp"
"strconv"
"strings"
"time"
)
// Web search for the assistant, when the user turned «Поиск» on: DuckDuckGo's
// plain pages, no key and nothing to install. The light page first, the
// HTML one if that fails; DuckDuckGo turns away too many requests from one
// address, and then the assistant is told so.
type searchResult struct {
Title string `json:"title"`
URL string `json:"url"`
Snippet string `json:"snippet"`
}
var (
reLiteLink = regexp.MustCompile(`(?s)<a[^>]+href="([^"]+)"[^>]*class='result-link'[^>]*>(.*?)</a>`)
reLiteSnippet = regexp.MustCompile(`(?s)<td class='result-snippet'>(.*?)</td>`)
reHTMLLink = regexp.MustCompile(`(?s)<a[^>]+class="result__a"[^>]+href="([^"]+)"[^>]*>(.*?)</a>`)
reHTMLSnippet = regexp.MustCompile(`(?s)<a[^>]+class="result__snippet"[^>]*>(.*?)</a>`)
reTags = regexp.MustCompile(`<[^>]+>`)
)
// text is a bit of a results page as plain text.
func text(s string) string {
return strings.Join(strings.Fields(html.UnescapeString(reTags.ReplaceAllString(s, ""))), " ")
}
// realURL takes DuckDuckGo's redirect (/l/?uddg=…) back to the address.
func realURL(h string) string {
h = html.UnescapeString(h)
if strings.HasPrefix(h, "//") {
h = "https:" + h
}
if u, err := url.Parse(h); err == nil {
if t := u.Query().Get("uddg"); t != "" {
return t
}
}
return h
}
// parseDDG reads the results of the light or the HTML page.
func parseDDG(page string) []searchResult {
links, snips := reLiteLink.FindAllStringSubmatch(page, -1), reLiteSnippet.FindAllStringSubmatch(page, -1)
if len(links) == 0 {
links, snips = reHTMLLink.FindAllStringSubmatch(page, -1), reHTMLSnippet.FindAllStringSubmatch(page, -1)
}
var out []searchResult
for i, l := range links {
r := searchResult{Title: text(l[2]), URL: realURL(l[1])}
if i < len(snips) {
r.Snippet = text(snips[i][1])
}
// Ads lead back through DuckDuckGo itself: left out.
if strings.Contains(r.URL, "duckduckgo.com/y.js") || r.Title == "" {
continue
}
out = append(out, r)
}
return out
}
var searchClient = &http.Client{Timeout: 15 * time.Second}
// The engine is the admins' choice (Настройки › ИИ-ассистент): DuckDuckGo,
// a SearXNG of one's own, Brave Search or Tavily; when the chosen one fails,
// DuckDuckGo answers instead.
const (
cfgSearchEngine = "ai_search"
cfgSearchURL = "ai_search_url"
cfgSearchKey = "ai_search_key"
)
type searchConf struct{ Engine, URL, Key string }
func (s *Server) searchConf() searchConf {
e, _ := s.db.GetConfig(cfgSearchEngine)
u, _ := s.db.GetConfig(cfgSearchURL)
k, _ := s.db.GetConfig(cfgSearchKey)
return searchConf{Engine: e, URL: strings.TrimRight(u, "/"), Key: k}
}
var (
braveURL = "https://api.search.brave.com/res/v1/web/search"
tavilyURL = "https://api.tavily.com/search"
ddgSearch = webSearch
)
// engineName is how the engine is called to people and to the model.
func engineName(e string) string {
switch e {
case "searxng":
return "SearXNG"
case "brave":
return "Brave Search"
case "tavily":
return "Tavily"
}
return "DuckDuckGo"
}
// searchWith asks the engine c names; DuckDuckGo if that one fails. It
// tells which engine the results came from (and why, when it was not the
// chosen one).
func searchWith(ctx context.Context, c searchConf, q string, n int) ([]searchResult, string, error) {
var rs []searchResult
var err error
switch c.Engine {
case "searxng":
rs, err = searchSearx(ctx, c, q)
case "brave":
rs, err = searchBrave(ctx, c, q, n)
case "tavily":
rs, err = searchTavily(ctx, c, q, n)
default:
rs, err = ddgSearch(ctx, q, n)
return rs, "DuckDuckGo", err
}
if err != nil || len(rs) == 0 {
why := "ничего не нашёл"
if err != nil {
why = err.Error()
}
log.Printf("search: %s failed (%s), DuckDuckGo instead", engineName(c.Engine), why)
rs, err = ddgSearch(ctx, q, n)
return rs, "DuckDuckGo (" + engineName(c.Engine) + " не ответил: " + why + ")", err
}
if len(rs) > n {
rs = rs[:n]
}
return rs, engineName(c.Engine), nil
}
// searchPrompt tells the model it can search, and with what.
func searchPrompt(c searchConf) string {
return "\nПоиск в интернете включён (поисковик: " + engineName(c.Engine) + "): блок ```search\nзапрос\n``` - панель найдёт и пришлёт результаты " +
"(заголовок, адрес, отрывок) следующим сообщением, с пометкой, через какой поисковик они пришли.\n" +
"Ищи активно: свежие данные, версии, документация, тексты ошибок, всё, в чём не уверен, - сначала поиск, потом ответ со ссылками на найденное. " +
"Чтобы прочитать найденную страницу целиком (или адрес, который дал пользователь), - блок ```fetch\nhttps://адрес\n```: придёт её текст. " +
"Одно действие за ответ. Ссылки не выдумывай."
}
// getJSON fetches url (with the headers) into v.
func getJSON(ctx context.Context, method, url string, body io.Reader, hdr map[string]string, v any) error {
req, err := http.NewRequestWithContext(ctx, method, url, body)
if err != nil {
return err
}
req.Header.Set("Accept", "application/json")
for k, val := range hdr {
req.Header.Set(k, val)
}
resp, err := searchClient.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode/100 != 2 {
return errors.New("поисковик ответил " + resp.Status)
}
return json.NewDecoder(io.LimitReader(resp.Body, 4<<20)).Decode(v)
}
// searchSearx: a SearXNG with JSON output on (search.formats: json).
func searchSearx(ctx context.Context, c searchConf, q string) ([]searchResult, error) {
if c.URL == "" {
return nil, errors.New("не задан адрес SearXNG")
}
var out struct {
Results []struct{ Title, URL, Content string } `json:"results"`
}
if err := getJSON(ctx, http.MethodGet, c.URL+"/search?format=json&q="+url.QueryEscape(q), nil, nil, &out); err != nil {
return nil, err
}
var rs []searchResult
for _, r := range out.Results {
rs = append(rs, searchResult{Title: r.Title, URL: r.URL, Snippet: text(r.Content)})
}
return rs, nil
}
func searchBrave(ctx context.Context, c searchConf, q string, n int) ([]searchResult, error) {
var out struct {
Web struct {
Results []struct{ Title, URL, Description string } `json:"results"`
} `json:"web"`
}
err := getJSON(ctx, http.MethodGet, braveURL+"?count="+strconv.Itoa(n)+"&q="+url.QueryEscape(q), nil,
map[string]string{"X-Subscription-Token": c.Key}, &out)
if err != nil {
return nil, err
}
var rs []searchResult
for _, r := range out.Web.Results {
rs = append(rs, searchResult{Title: text(r.Title), URL: r.URL, Snippet: text(r.Description)})
}
return rs, nil
}
func searchTavily(ctx context.Context, c searchConf, q string, n int) ([]searchResult, error) {
b, _ := json.Marshal(map[string]any{"query": q, "max_results": n})
var out struct {
Results []struct{ Title, URL, Content string } `json:"results"`
}
err := getJSON(ctx, http.MethodPost, tavilyURL, strings.NewReader(string(b)),
map[string]string{"Authorization": "Bearer " + c.Key, "Content-Type": "application/json"}, &out)
if err != nil {
return nil, err
}
var rs []searchResult
for _, r := range out.Results {
rs = append(rs, searchResult{Title: r.Title, URL: r.URL, Snippet: r.Content})
}
return rs, nil
}
// webSearch asks DuckDuckGo for q: up to n results.
func webSearch(ctx context.Context, q string, n int) ([]searchResult, error) {
var last error
for _, base := range []string{"https://lite.duckduckgo.com/lite/?q=", "https://html.duckduckgo.com/html/?q="} {
req, _ := http.NewRequestWithContext(ctx, http.MethodGet, base+url.QueryEscape(q), nil)
req.Header.Set("User-Agent", "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0 Safari/537.36")
req.Header.Set("Accept-Language", "ru,en;q=0.8")
resp, err := searchClient.Do(req)
if err != nil {
last = err
continue
}
b, _ := io.ReadAll(io.LimitReader(resp.Body, 2<<20))
resp.Body.Close()
rs := parseDDG(string(b))
if resp.StatusCode == http.StatusOK && len(rs) > 0 {
if len(rs) > n {
rs = rs[:n]
}
return rs, nil
}
if resp.StatusCode == http.StatusOK {
last = errors.New("ничего не нашлось")
} else {
last = errors.New("поисковик не ответил (" + resp.Status + "), возможно, временно ограничил запросы")
}
}
return nil, last
}
// POST /api/panel/ai/search - {q}: the results, for a user with the
// assistant on (the «Поиск» switch is the page's: it lets the model ask).
func (s *Server) handleAISearch(w http.ResponseWriter, r *http.Request) {
a := s.currentAdmin(r)
if !s.aiOn(a.ID) {
writeJSON(w, map[string]any{"ok": false, "error": "ассистент выключен в профиле"})
return
}
var in struct {
Q string `json:"q"`
}
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 4<<10)).Decode(&in); err != nil || strings.TrimSpace(in.Q) == "" {
writeJSON(w, map[string]any{"ok": false, "error": "пустой запрос"})
return
}
rs, used, err := searchWith(r.Context(), s.searchConf(), strings.TrimSpace(in.Q), 6)
if err != nil {
writeJSON(w, map[string]any{"ok": false, "error": err.Error(), "engine": used})
return
}
writeJSON(w, map[string]any{"ok": true, "data": rs, "engine": used})
}
+107
View File
@@ -0,0 +1,107 @@
package server
import (
"context"
"encoding/json"
"fmt"
"net/http"
"net/http/httptest"
"os"
"strings"
"testing"
)
// The results of DuckDuckGo's light page: titles, real addresses (not its
// redirects), snippets without tags.
func TestParseDDGLite(t *testing.T) {
b, err := os.ReadFile("testdata/ddg_lite.html")
if err != nil {
t.Fatal(err)
}
rs := parseDDG(string(b))
if len(rs) < 5 {
t.Fatalf("results: %d", len(rs))
}
r := rs[0]
if r.URL != "https://go.dev/doc/database/cancel-operations" || !strings.Contains(r.Title, "Canceling in-progress operations") ||
!strings.Contains(r.Snippet, "Learn how to use context.Context") || strings.Contains(r.Snippet, "<b>") {
t.Fatalf("first: %+v", r)
}
}
// Each search engine of the settings, and DuckDuckGo when the chosen one
// fails.
func TestSearchProviders(t *testing.T) {
var gotKey, gotQ string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch {
case r.URL.Path == "/search" && r.URL.Query().Get("format") == "json": // SearXNG
gotQ = r.URL.Query().Get("q")
fmt.Fprint(w, `{"results":[{"title":"SX","url":"https://a.example/","content":"from searx"}]}`)
case r.URL.Path == "/res/v1/web/search": // Brave
gotKey, gotQ = r.Header.Get("X-Subscription-Token"), r.URL.Query().Get("q")
fmt.Fprint(w, `{"web":{"results":[{"title":"BR","url":"https://b.example/","description":"from <strong>brave</strong>"}]}}`)
case r.URL.Path == "/search" && r.Method == http.MethodPost: // Tavily
var in map[string]any
_ = json.NewDecoder(r.Body).Decode(&in)
gotKey, gotQ = r.Header.Get("Authorization"), in["query"].(string)
fmt.Fprint(w, `{"results":[{"title":"TV","url":"https://t.example/","content":"from tavily"}]}`)
default:
http.Error(w, "no", 500)
}
}))
defer srv.Close()
braveURL, tavilyURL = srv.URL+"/res/v1/web/search", srv.URL+"/search"
ctx := context.Background()
rs, used, err := searchWith(ctx, searchConf{Engine: "searxng", URL: srv.URL}, "q1", 6)
if err != nil || len(rs) != 1 || rs[0].Title != "SX" || gotQ != "q1" || used != "SearXNG" {
t.Fatalf("searxng: %+v %v", rs, err)
}
rs, _, err = searchWith(ctx, searchConf{Engine: "brave", Key: "bk"}, "q2", 6)
if err != nil || rs[0].Snippet != "from brave" || gotKey != "bk" || gotQ != "q2" {
t.Fatalf("brave: %+v %v %q", rs, err, gotKey)
}
rs, used, err = searchWith(ctx, searchConf{Engine: "tavily", Key: "tk"}, "q3", 6)
if err != nil || rs[0].Title != "TV" || gotKey != "Bearer tk" || gotQ != "q3" || used != "Tavily" {
t.Fatalf("tavily: %+v %v %q", rs, err, gotKey)
}
// The chosen one fails: DuckDuckGo instead (stood in for here).
ddgSearch = func(ctx context.Context, q string, n int) ([]searchResult, error) {
return []searchResult{{Title: "DDG"}}, nil
}
rs, used, err = searchWith(ctx, searchConf{Engine: "searxng", URL: srv.URL + "/broken"}, "q4", 6)
if err != nil || rs[0].Title != "DDG" || !strings.Contains(used, "DuckDuckGo") || !strings.Contains(used, "SearXNG не ответил") {
t.Fatalf("fallback: %+v %v", rs, err)
}
}
// The chosen engine's limit spent (429 and the like): DuckDuckGo answers.
func TestSearchLimitFallsBack(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
http.Error(w, `{"error":"rate limit"}`, http.StatusTooManyRequests)
}))
defer srv.Close()
old := ddgSearch
defer func() { ddgSearch = old }()
ddgSearch = func(ctx context.Context, q string, n int) ([]searchResult, error) {
return []searchResult{{Title: "DDG " + q}}, nil
}
braveURL, tavilyURL = srv.URL, srv.URL
for _, e := range []string{"brave", "tavily", "searxng"} {
rs, _, err := searchWith(context.Background(), searchConf{Engine: e, Key: "k", URL: srv.URL}, "x", 6)
if err != nil || len(rs) != 1 || rs[0].Title != "DDG x" {
t.Fatalf("%s: %+v %v", e, rs, err)
}
}
}
// The model is told which engine it searches with.
func TestSearchPromptNamesEngine(t *testing.T) {
if p := searchPrompt(searchConf{Engine: "tavily"}); !strings.Contains(p, "Tavily") || strings.Contains(p, "DuckDuckGo") {
t.Fatalf("tavily: %s", p)
}
if p := searchPrompt(searchConf{}); !strings.Contains(p, "DuckDuckGo") {
t.Fatalf("default: %s", p)
}
}
+272 -27
View File
@@ -1,7 +1,9 @@
package server
import (
"crypto/sha256"
"embed"
"encoding/hex"
"html/template"
"io/fs"
"log"
@@ -9,6 +11,8 @@ import (
"net/http"
"strconv"
"strings"
"sync"
"sync/atomic"
texttemplate "text/template"
)
@@ -24,7 +28,8 @@ type Options struct {
// nginx TLS); set false only for plain-HTTP local testing.
SecureCookies bool
// TunnelAddr is the listen address for the direct TCP+UDP forward tunnel
// (default ":8446"). Empty disables direct forwarding.
// Empty disables direct forwarding. The port can be overridden from the
// settings page.
TunnelAddr string
}
@@ -33,12 +38,33 @@ type Server struct {
db *DB
hub *Hub
fwd *forwardManager
tun *tunnelHub
udp *udpRelay
ufw *ufwManager
tmpl *template.Template
limiter *loginLimiter
distDir string
secureCookies bool
installTmpl *texttemplate.Template
// Direct tunnel: bind host and port of the -tunnel address, and the port
// currently listened on (the settings override, else the default).
// tunnelDefault 0 means the tunnel is disabled.
tunnelBind string
tunnelDefault int
tunnelMu sync.Mutex // serializes port changes
tunnelPort atomic.Int64
stats statCache // last load sample per agent, for the status page
relays speedRelays // speed tests that run through the browser
meters speedMeters // byte counts of the speed tests under way
xfers xfers // files on their way between a browser and an agent
ship shipping // files on their way from one machine to another
viaDirectA sync.Map // agent id -> true while it reaches the server at the direct address
challenges challenges // passkey challenges handed out
wsAgents wsAgents // agents on their WebSocket now
pinMisses pinMisses // wrong PINs per session
assets map[string]string // static file -> hash of its content
}
// New builds the server, parsing embedded templates.
@@ -46,7 +72,15 @@ func New(opts Options) (*Server, error) {
if err := opts.DB.ensureDefaults(); err != nil {
return nil, err
}
tmpl, err := template.ParseFS(webFS, "web/templates/*.html")
mainHost = func() string { return directHost(opts.DB.HostAddress()) }
staticFS, _ := fs.Sub(webFS, "web/static")
assets := assetHashes(staticFS)
funcs := template.FuncMap{
// asset is a static file's URL with its content hash, so a new
// release is a new URL that no browser or proxy has cached.
"asset": func(name string) string { return "/static/" + name + "?v=" + assets[name] },
}
tmpl, err := template.New("").Funcs(funcs).ParseFS(webFS, "web/templates/*.html")
if err != nil {
return nil, err
}
@@ -65,26 +99,88 @@ func New(opts Options) (*Server, error) {
db: opts.DB,
hub: hub,
fwd: newForwardManager(hub, tun, udp),
ufw: newUfwManager(opts.DB, portFromAddr(opts.TunnelAddr)),
tun: tun,
udp: udp,
tmpl: tmpl,
limiter: newLoginLimiter(),
distDir: opts.DistDir,
secureCookies: opts.SecureCookies,
installTmpl: installTmpl,
assets: assets,
}
// Start the direct forward tunnel (TCP mux + UDP) on 8446.
s.ufw = newUfwManager(opts.DB, s.TunnelPort)
// Start the direct forward tunnel (TCP mux + UDP), on the port from the
// settings page if one is set there.
if opts.TunnelAddr != "" {
if err := tun.listen(opts.TunnelAddr); err != nil {
log.Printf("tunnel tcp listen %s: %v", opts.TunnelAddr, err)
s.tunnelBind, s.tunnelDefault = splitAddr(opts.TunnelAddr)
}
if err := udp.listen(opts.TunnelAddr); err != nil {
log.Printf("tunnel udp listen %s: %v", opts.TunnelAddr, err)
if s.tunnelDefault > 0 {
port := s.tunnelDefault
if p := s.db.TunnelPort(); p > 0 {
port = p
}
s.tunnelPort.Store(int64(port))
addr := s.tunnelListenAddr(port)
if err := tun.listen(addr); err != nil {
log.Printf("tunnel tcp listen %s: %v", addr, err)
}
if err := udp.listen(addr); err != nil {
log.Printf("tunnel udp listen %s: %v", addr, err)
}
}
s.restoreForwards()
go s.ufw.sync()
return s, nil
}
// TunnelPort is the port the direct tunnel listens on (0 = disabled).
func (s *Server) TunnelPort() int { return int(s.tunnelPort.Load()) }
func (s *Server) tunnelListenAddr(port int) string {
return net.JoinHostPort(s.tunnelBind, strconv.Itoa(port))
}
// tunnelAdvert is the tunnel address handed to agents as "host:port". The host
// is the one set in Settings, else the direct address's (the main one may be
// behind a proxy that carries no raw TCP), else empty: agents then use the
// panel's host.
func (s *Server) tunnelAdvert() string {
port := s.TunnelPort()
if port == 0 {
return ""
}
host := s.db.TunnelHost()
if host == "" {
host = directHost(s.db.DirectAddress())
}
return net.JoinHostPort(host, strconv.Itoa(port))
}
// setTunnelPort moves the direct tunnel to another port. Connected agents are
// dropped and redial the new port once their next poll tells them about it.
func (s *Server) setTunnelPort(port int) error {
s.tunnelMu.Lock()
defer s.tunnelMu.Unlock()
old := s.TunnelPort()
if old == 0 || port == old {
return nil
}
addr := s.tunnelListenAddr(port)
if err := s.tun.listen(addr); err != nil {
return err
}
if err := s.udp.listen(addr); err != nil {
// Keep TCP and UDP on the same port: go back to the old one.
if rerr := s.tun.listen(s.tunnelListenAddr(old)); rerr != nil {
log.Printf("tunnel tcp listen %s: %v", s.tunnelListenAddr(old), rerr)
}
return err
}
s.tunnelPort.Store(int64(port))
go s.ufw.sync()
return nil
}
// restoreForwards re-opens the host listeners for all saved forward rules.
func (s *Server) restoreForwards() {
rules, err := s.db.ListForwardRules()
@@ -93,6 +189,12 @@ func (s *Server) restoreForwards() {
return
}
for _, r := range rules {
// Rules of an agent deleted by an older version: drop them.
if a, err := s.db.GetAgent(r.AgentID); err == nil && a == nil {
log.Printf("forward rule %d: agent %s is gone, removing", r.ID, r.AgentID)
s.removeForward(r.ID)
continue
}
if err := s.fwd.open(r); err != nil {
log.Printf("restore forward rule %d (%s:%d): %v", r.ID, r.ListenAddr, r.HostPort, err)
}
@@ -105,7 +207,7 @@ func (s *Server) Handler() http.Handler {
// Static assets (xterm.js etc.) from the embedded FS.
staticFS, _ := fs.Sub(webFS, "web/static")
mux.Handle("GET /static/", http.StripPrefix("/static/", http.FileServer(http.FS(staticFS))))
mux.Handle("GET /static/", http.StripPrefix("/static/", staticHandler(staticFS, s.assets)))
// Public: install script + agent binary downloads.
mux.HandleFunc("GET /install.sh", s.handleInstallScript)
@@ -120,35 +222,98 @@ func (s *Server) Handler() http.Handler {
mux.HandleFunc("GET /login", s.handleLoginForm)
mux.HandleFunc("POST /login", s.handleLoginSubmit)
mux.HandleFunc("POST /logout", s.handleLogout)
mux.HandleFunc("POST /api/login/key/begin", s.handleKeyLoginBegin)
mux.HandleFunc("POST /api/login/key/finish", s.handleKeyLoginFinish)
// Panel (admin-only).
mux.Handle("GET /{$}", s.requireAdmin(s.handleDashboard))
mux.Handle("GET /agents/{id}/terminal", s.requireAdmin(s.handleTerminalPage))
mux.Handle("GET /agents/{id}/files", s.requireAdmin(s.handleFilesPage))
mux.Handle("GET /workspace", s.requireAdmin(s.handleShell))
mux.Handle("GET /terminal", s.requireAdmin(s.handleShell)) // the old address
mux.Handle("GET /agents/{id}/terminal", s.requireAdmin(s.handleShell))
mux.Handle("GET /agents/{id}/{kind}", s.requireAdmin(s.handleShell)) // files
mux.Handle("GET /agents/{id}/forwards", s.requireAdmin(s.handleForwardsPage))
mux.Handle("GET /agents/{id}/status", s.requireAdmin(s.handleStatusPage))
mux.Handle("GET /agents/{id}/edit", s.requireAdmin(s.handleAgentEdit))
mux.Handle("POST /agents/{id}/rename", s.requireAdmin(s.handleAgentRename))
mux.Handle("POST /agents/{id}/update", s.requireAdmin(s.handleAgentUpdate))
mux.Handle("POST /agents/{id}/user", s.requireAdmin(s.handleAgentSetUser))
mux.Handle("POST /agents/{id}/transport", s.requireAdmin(s.handleAgentTransport))
mux.Handle("POST /agents/{id}/users/add", s.requireAdmin(s.handleAgentAddUser))
mux.Handle("POST /agents/{id}/delete", s.requireAdmin(s.handleAgentDelete))
mux.Handle("GET /settings", s.requireAdmin(s.handleSettings))
mux.Handle("POST /settings/host", s.requireAdmin(s.handleSetHost))
mux.Handle("POST /settings/secret/rotate", s.requireAdmin(s.handleRotateSecret))
mux.Handle("POST /settings/admins/add", s.requireAdmin(s.handleAddAdmin))
mux.Handle("POST /settings/admins/delete", s.requireAdmin(s.handleDeleteAdmin))
mux.Handle("POST /settings/ufw/toggle", s.requireAdmin(s.handleUfwToggle))
mux.Handle("POST /settings/ufw/revoke", s.requireAdmin(s.handleUfwRevoke))
mux.Handle("POST /agents/{id}/delete", s.adminsOnly(s.handleAgentDelete))
mux.Handle("GET /settings", s.adminsOnly(s.handleSettings))
mux.Handle("POST /settings/host", s.adminsOnly(s.handleSetHost))
mux.Handle("POST /settings/tunnel", s.adminsOnly(s.handleSetTunnel))
mux.Handle("POST /settings/secret/rotate", s.adminsOnly(s.handleRotateSecret))
mux.Handle("POST /settings/install/code", s.adminsOnly(s.handleInstallCode))
mux.Handle("POST /settings/admins/add", s.adminsOnly(s.handleAddAdmin))
mux.Handle("POST /settings/admins/delete", s.adminsOnly(s.handleDeleteAdmin))
mux.Handle("GET /api/panel/users/{id}/agents", s.adminsOnly(s.handleUserAgents))
mux.Handle("POST /api/panel/users/{id}", s.adminsOnly(s.handleUserEdit))
mux.Handle("POST /settings/admins/keys/reset", s.adminsOnly(s.handleResetAdminKeys))
mux.Handle("GET /api/panel/account", s.requireAdmin(s.handleAccount))
mux.Handle("GET /api/panel/account/prefs", s.requireAdmin(s.handlePrefs))
mux.Handle("POST /api/panel/account/prefs", s.requireAdmin(s.handlePrefsSet))
mux.Handle("GET /api/panel/ai", s.requireAdmin(s.handleAIState))
mux.Handle("GET /api/panel/ssh/hosts", s.requireAdmin(s.handleSSHHosts))
mux.Handle("POST /api/panel/ssh/hosts", s.requireAdmin(s.handleSSHHostAdd))
mux.Handle("POST /api/panel/ssh/hosts/{hid}/delete", s.requireAdmin(s.handleSSHHostDelete))
mux.Handle("POST /api/panel/ai/toggle", s.requireAdmin(s.handleAIToggle))
mux.Handle("POST /api/panel/ai/chat", s.requireAdmin(s.handleAIChat))
mux.Handle("GET /api/panel/ai/chats", s.requireAdmin(s.handleAIChats))
mux.Handle("POST /api/panel/ai/chats", s.requireAdmin(s.handleAIChatNew))
mux.Handle("GET /api/panel/ai/chats/{cid}", s.requireAdmin(s.handleAIHistory))
mux.Handle("POST /api/panel/ai/chats/{cid}/delete", s.requireAdmin(s.handleAIChatDelete))
mux.Handle("POST /api/panel/ai/note", s.requireAdmin(s.handleAINote))
mux.Handle("POST /api/panel/ai/search", s.requireAdmin(s.handleAISearch))
mux.Handle("POST /api/panel/ai/fetch", s.requireAdmin(s.handleAIFetch))
mux.Handle("POST /api/panel/ai/image", s.requireAdmin(s.handleAIImage))
mux.Handle("GET /api/panel/ai/image/{job}", s.requireAdmin(s.handleAIImageJob))
mux.Handle("POST /settings/ai", s.adminsOnly(s.handleSetAI))
mux.Handle("POST /settings/db", s.adminsOnly(s.handleSetDB))
mux.Handle("POST /settings/db/restart", s.adminsOnly(s.handleRestart))
mux.Handle("POST /api/panel/account/keys/begin", s.requireAdmin(s.handleKeyRegisterBegin))
mux.Handle("POST /api/panel/account/keys/finish", s.requireAdmin(s.handleKeyRegisterFinish))
mux.Handle("POST /api/panel/account/keys/{id}/delete", s.requireAdmin(s.handleKeyDelete))
mux.Handle("POST /api/panel/account/password", s.requireAdmin(s.handlePasswordToggle))
mux.Handle("POST /api/panel/account/pin", s.requireAdmin(s.handlePinSet))
mux.Handle("GET /api/panel/pin", s.requireAdmin(s.handlePinState))
mux.Handle("POST /api/panel/pin/check", s.requireAdmin(s.handlePinCheck))
mux.Handle("POST /api/panel/account/password/change", s.requireAdmin(s.handlePasswordChange))
mux.Handle("POST /settings/ufw/toggle", s.adminsOnly(s.handleUfwToggle))
mux.Handle("POST /settings/ufw/add", s.adminsOnly(s.handleUfwAdd))
mux.Handle("POST /settings/ufw/revoke", s.adminsOnly(s.handleUfwRevoke))
// Panel terminal over HTTP long-polling (admin-only).
mux.Handle("POST /api/panel/term/{agentID}/open", s.requireAdmin(s.handlePanelOpen))
mux.Handle("GET /api/panel/term/{sid}/out", s.requireAdmin(s.handlePanelOut))
mux.Handle("POST /api/panel/term/{sid}/in", s.requireAdmin(s.handlePanelIn))
mux.Handle("GET /api/panel/term/{sid}/ws", s.requireAdmin(s.handlePanelTermWS))
mux.Handle("POST /api/panel/term/{sid}/close", s.requireAdmin(s.handlePanelClose))
// Panel file manager (admin-only).
mux.Handle("GET /api/panel/agents/{id}/fs/download", s.requireAdmin(s.handleFSDownload))
mux.Handle("POST /api/panel/agents/{id}/fs/upload", s.requireAdmin(s.handleFSUploadStart))
mux.Handle("PUT /api/panel/fs/xfer/{xid}/{i}", s.requireAdmin(s.handleXferPut))
mux.Handle("POST /api/panel/fs/xfer/{xid}/cancel", s.requireAdmin(s.handleXferCancel))
mux.Handle("GET /api/panel/fs/xfer/{xid}/result", s.requireAdmin(s.handleXferResult))
mux.Handle("POST /api/panel/agents/{id}/fs/ship", s.requireAdmin(s.handleFSShip))
mux.Handle("GET /api/panel/fs/job/{jid}", s.requireAdmin(s.handleShipJob))
mux.Handle("POST /api/panel/fs/job/{jid}/cancel", s.requireAdmin(s.handleShipCancel))
mux.Handle("POST /api/panel/agents/{id}/fs/{op}", s.requireAdmin(s.handleFSOp))
mux.Handle("POST /api/panel/agents/{id}/sshfs/open", s.requireAdmin(s.handleSSHFSOpen))
mux.Handle("GET /api/panel/agents/{id}/sshfs/{conn}/download", s.requireAdmin(s.handleSSHFSDownload))
mux.Handle("POST /api/panel/agents/{id}/sshfs/{conn}/upload", s.requireAdmin(s.handleSSHFSUpload))
mux.Handle("POST /api/panel/agents/{id}/sshfs/{conn}/{op}", s.requireAdmin(s.handleSSHFSOp))
// Machine status (admin).
mux.Handle("GET /api/panel/agents/presence", s.requireAdmin(s.handlePresence))
mux.Handle("GET /api/panel/agents/{id}/stat", s.requireAdmin(s.handleAgentStat))
mux.Handle("POST /api/panel/agents/{id}/speed", s.requireAdmin(s.handleAgentSpeed))
mux.Handle("GET /api/panel/agents/{id}/speed/progress", s.requireAdmin(s.handleSpeedProgress))
mux.Handle("GET /api/panel/ping", s.requireAdmin(s.handlePanelPing))
mux.Handle("POST /api/panel/agents/{id}/speed/relay", s.requireAdmin(s.handleSpeedRelayOpen))
mux.Handle("POST /api/panel/agents/{id}/speed/down", s.requireAdmin(s.handleSpeedRelayFeed))
mux.Handle("GET /api/panel/agents/{id}/speed/up", s.requireAdmin(s.handleSpeedRelayDrain))
// Panel port forwarding (admin-only).
mux.Handle("POST /agents/{id}/forwards/add", s.requireAdmin(s.handleForwardAdd))
@@ -157,10 +322,21 @@ func (s *Server) Handler() http.Handler {
// Agent API (token/secret auth inside handlers).
mux.HandleFunc("POST /api/agent/enroll", s.handleEnroll)
mux.HandleFunc("GET /api/agent/poll", s.handlePoll)
mux.HandleFunc("GET /api/agent/ws", s.handleAgentWS)
mux.HandleFunc("GET /api/agent/config", s.handleAgentConfig)
mux.HandleFunc("GET /api/agent/term/{sid}/in", s.handleTermIn)
mux.HandleFunc("POST /api/agent/term/{sid}/out", s.handleTermOut)
mux.HandleFunc("GET /api/agent/task/{id}", s.handleTaskFetch)
mux.HandleFunc("POST /api/agent/task/{id}/result", s.handleTaskResult)
mux.HandleFunc("GET /api/agent/speed/down", s.handleSpeedDown)
mux.HandleFunc("POST /api/agent/speed/up", s.handleSpeedUp)
mux.HandleFunc("GET /api/agent/speed/relay/down", s.handleSpeedRelayDown)
mux.HandleFunc("POST /api/agent/speed/relay/up", s.handleSpeedRelayUp)
mux.HandleFunc("GET /api/agent/fs/xfer/{xid}/{i}", s.handleAgentXferGet)
mux.HandleFunc("POST /api/agent/fs/xfer/{xid}/{i}", s.handleAgentXferPost)
mux.HandleFunc("POST /api/agent/fs/job/{jid}/progress", s.handleShipProgress)
mux.HandleFunc("POST /api/agent/fs/relay/{rid}/{i}", s.handleRelayPost)
mux.HandleFunc("GET /api/agent/fs/relay/{rid}/{i}", s.handleRelayGet)
return s.logRequests(mux)
}
@@ -177,6 +353,36 @@ func (s *Server) requireAdmin(h http.HandlerFunc) http.Handler {
http.Redirect(w, r, "/login", http.StatusSeeOther)
return
}
// A user who is not an admin uses only the machines given to them:
// any other is as if it were not there.
if !admin.IsAdmin() {
p := r.URL.Path
if strings.HasPrefix(p, "/agents/") || strings.HasPrefix(p, "/api/panel/agents/") || strings.HasPrefix(p, "/api/panel/term/") {
id := r.PathValue("id")
if id == "" {
id = r.PathValue("agentID")
}
if id != "" && id != "presence" && !s.db.CanUseAgent(admin, id) {
http.NotFound(w, r)
return
}
}
}
h(w, r)
})
}
// adminsOnly lets only admins through: the settings, users, the firewall.
func (s *Server) adminsOnly(h http.HandlerFunc) http.Handler {
return s.requireAdmin(func(w http.ResponseWriter, r *http.Request) {
if !s.currentAdmin(r).IsAdmin() {
if r.Method == http.MethodGet && !strings.HasPrefix(r.URL.Path, "/api/") {
http.Redirect(w, r, "/", http.StatusSeeOther)
return
}
http.Error(w, "только для администраторов", http.StatusForbidden)
return
}
h(w, r)
})
}
@@ -202,17 +408,15 @@ func (s *Server) render(w http.ResponseWriter, name string, data map[string]any)
}
}
// portFromAddr extracts the numeric port from a "host:port" or ":port" address.
func portFromAddr(addr string) int {
if addr == "" {
return 0
}
_, p, err := net.SplitHostPort(addr)
// splitAddr splits a "host:port" or ":port" listen address. The port is 0 when
// the address has none.
func splitAddr(addr string) (host string, port int) {
h, p, err := net.SplitHostPort(addr)
if err != nil {
return 0
return "", 0
}
n, _ := strconv.Atoi(p)
return n
return h, n
}
func clientIP(r *http.Request) string {
@@ -232,3 +436,44 @@ func clientIP(r *http.Request) string {
}
return host
}
// assetHashes hashes every static file, for asset URLs and ETags.
func assetHashes(fsys fs.FS) map[string]string {
out := map[string]string{}
_ = fs.WalkDir(fsys, ".", func(p string, d fs.DirEntry, err error) error {
if err != nil || d.IsDir() {
return nil
}
if data, err := fs.ReadFile(fsys, p); err == nil {
sum := sha256.Sum256(data)
out[p] = hex.EncodeToString(sum[:8])
}
return nil
})
return out
}
// staticHandler serves the embedded assets. Pages ask for them by URLs that
// carry the content hash (?v=): such a file never changes and is kept for a
// year, by the browser and by a proxy in front such as Cloudflare. Asked for
// without it, a file is checked every time against its ETag.
func staticHandler(fsys fs.FS, hashes map[string]string) http.Handler {
files := http.FileServer(http.FS(fsys))
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// The JS tests sit beside the scripts but are not served.
if strings.HasSuffix(r.URL.Path, ".test.js") {
http.NotFound(w, r)
return
}
h := hashes[strings.TrimPrefix(r.URL.Path, "/")]
if h != "" {
w.Header().Set("ETag", `"`+h+`"`)
}
if v := r.URL.Query().Get("v"); h != "" && v == h {
w.Header().Set("Cache-Control", "public, max-age=31536000, immutable")
} else {
w.Header().Set("Cache-Control", "no-cache")
}
files.ServeHTTP(w, r)
})
}
+496
View File
@@ -0,0 +1,496 @@
package server
import (
"context"
"encoding/json"
"net/http"
"path"
"strconv"
"strings"
"sync"
"time"
"github.com/x-grk/longpolling-remote-access/internal/proto"
)
// Sending a file or folder to another machine. The sending agent packs it as
// a tar stream; to another agent the stream goes in pieces through a relay
// here, a few pieces held at most, and the receiving agent unpacks it as it
// comes. To an SSH host the sending agent writes it there itself. The panel
// follows a job: how far the sender is, and the result.
// relay carries one stream between two agents, in numbered pieces.
type relay struct {
id, from, to string
mu sync.Mutex
changed chan struct{}
chunks map[int][]byte
low int // pieces below it were taken
end int // the last piece's number, -1 until it came
err string
}
func (x *relay) notify() {
close(x.changed)
x.changed = make(chan struct{})
}
// await waits until ready() holds and returns true with x.mu held.
func (x *relay) await(ctx context.Context, d time.Duration, ready func() bool) bool {
t := time.NewTimer(d)
defer t.Stop()
for {
x.mu.Lock()
if ready() {
return true
}
ch := x.changed
x.mu.Unlock()
select {
case <-ch:
case <-t.C:
return false
case <-ctx.Done():
return false
}
}
}
func (x *relay) fail(msg string) {
x.mu.Lock()
if x.err == "" {
x.err = msg
x.chunks = nil
x.notify()
}
x.mu.Unlock()
}
type shipJob struct {
id string
mu sync.Mutex
changed chan struct{}
ver int
prog proto.ShipProgress
done bool
res proto.TaskResult
cancel context.CancelFunc
stopped bool
}
func (j *shipJob) update(fn func()) {
j.mu.Lock()
fn()
j.ver++
close(j.changed)
j.changed = make(chan struct{})
j.mu.Unlock()
}
type shipping struct {
mu sync.Mutex
relays map[string]*relay
jobs map[string]*shipJob
}
func (c *shipping) relay(id string) *relay {
c.mu.Lock()
defer c.mu.Unlock()
return c.relays[id]
}
func (c *shipping) job(id string) *shipJob {
c.mu.Lock()
defer c.mu.Unlock()
return c.jobs[id]
}
// shipReq is what the panel asks for: Src on this agent to Dir/Name on agent
// To, or on the SSH host.
type shipReq struct {
Src string `json:"src"`
Name string `json:"name"`
Dir string `json:"dir"`
Conflict string `json:"conflict"`
Move bool `json:"move"`
To string `json:"to"`
SSH *proto.SSHTarget `json:"ssh"`
// FromConn, ToConn: that side is an SSH host held by its agent for a
// file pane (its connection id); To is still the agent holding ToConn.
FromConn string `json:"from_conn"`
ToConn string `json:"to_conn"`
// More: other things of Src's folder sent along, under their own names.
More []string `json:"more"`
}
// POST /api/panel/agents/{id}/fs/ship - start sending; the answer names the
// job to follow.
func (s *Server) handleFSShip(w http.ResponseWriter, r *http.Request) {
from := r.PathValue("id")
var q shipReq
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 64<<10)).Decode(&q); err != nil {
writeJSON(w, proto.TaskResult{Error: "bad request"})
return
}
q.Name, q.Dir = strings.TrimSpace(q.Name), strings.TrimSpace(q.Dir)
if q.Src == "" || q.Name == "" || strings.Contains(q.Name, "/") || q.Dir == "" {
writeJSON(w, proto.TaskResult{Error: "не указано, что и куда"})
return
}
if !s.fsAgent(w, from, true) {
return
}
where := ""
if err := s.fillSaved(s.currentAdmin(r).ID, q.SSH); err != nil {
writeJSON(w, proto.TaskResult{Error: err.Error()})
return
}
if q.SSH != nil {
t := q.SSH
t.Host, t.User = strings.TrimSpace(t.Host), strings.TrimSpace(t.User)
if t.Port == 0 {
t.Port = 22
}
if t.Host == "" || t.User == "" || t.Port < 1 || t.Port > 65535 {
writeJSON(w, proto.TaskResult{Error: "нужны хост, порт и пользователь"})
return
}
where = "ssh:" + t.User + "@" + t.Host + ":" + strconv.Itoa(t.Port)
} else {
if q.To == "" && q.ToConn != "" {
q.To = from
}
if q.To == "" {
writeJSON(w, proto.TaskResult{Error: "не выбрана машина"})
return
}
if !s.db.CanUseAgent(s.currentAdmin(r), q.To) {
writeJSON(w, proto.TaskResult{Error: "нет доступа к той машине"})
return
}
if !s.fsAgent(w, q.To, true) {
return
}
where = "agent:" + q.To
}
ctx, cancel := context.WithTimeout(context.Background(), xferTimeout)
j := &shipJob{id: randToken()[:24], changed: make(chan struct{}), cancel: cancel}
s.ship.mu.Lock()
if s.ship.jobs == nil {
s.ship.jobs, s.ship.relays = map[string]*shipJob{}, map[string]*relay{}
}
s.ship.jobs[j.id] = j
s.ship.mu.Unlock()
if admin := s.currentAdmin(r); admin != nil {
op := "copy"
if q.Move {
op = "move"
}
s.db.Audit(admin.ID, from, "fs.ship."+op+":"+q.Src+" -> "+where+":"+strings.TrimSuffix(q.Dir, "/")+"/"+q.Name)
}
go s.runShip(ctx, j, from, q)
data, _ := json.Marshal(map[string]string{"job": j.id})
writeJSON(w, proto.TaskResult{OK: true, Data: data})
}
func (s *Server) runShip(ctx context.Context, j *shipJob, from string, q shipReq) {
defer j.cancel()
res := s.ship1(ctx, j, from, q)
if !res.OK && j.stopped {
res.Error = "отменено"
}
j.update(func() { j.done, j.res = true, res })
// The result waits a while for the panel to come for it.
time.AfterFunc(xferIdle, func() {
s.ship.mu.Lock()
delete(s.ship.jobs, j.id)
s.ship.mu.Unlock()
})
}
func oldAgent(res proto.TaskResult) proto.TaskResult {
if unknownOp(res) {
res.Error = "агент не умеет отправлять на другие машины - обновите его"
}
return res
}
func (s *Server) ship1(ctx context.Context, j *shipJob, from string, q shipReq) proto.TaskResult {
if len(q.More) > 0 && !s.shipsSeveral(from, q) {
// One at a time: an SSH address, or an agent from before 0.2.0.
var last proto.TaskResult
done, skipped := 0, 0
for i, p := range append([]string{q.Src}, q.More...) {
one := q
one.Src, one.More = p, nil
if i > 0 {
one.Name = path.Base(path.Clean(p))
}
if last = s.ship1(ctx, j, from, one); !last.OK {
return last
}
var d struct{ Done, Skipped int }
_ = json.Unmarshal(last.Data, &d)
done, skipped = done+d.Done, skipped+d.Skipped
}
data, _ := json.Marshal(map[string]int{"done": done, "skipped": skipped})
return proto.TaskResult{OK: true, Data: data}
}
args := proto.FSShipArgs{Src: q.Src, Name: q.Name, Dir: q.Dir, Conflict: q.Conflict, Job: j.id, More: q.More}
if q.SSH != nil {
args.SSH, args.Move = q.SSH, q.Move
res, err := s.waitTask(ctx, from, proto.TaskFSShipOut, args, xferTimeout)
if err != nil {
return proto.TaskResult{Error: err.Error()}
}
return oldAgent(res)
}
x := &relay{id: randToken()[:24], from: from, to: q.To, changed: make(chan struct{}), chunks: map[int][]byte{}, end: -1}
s.ship.mu.Lock()
s.ship.relays[x.id] = x
s.ship.mu.Unlock()
defer func() {
s.ship.mu.Lock()
delete(s.ship.relays, x.id)
s.ship.mu.Unlock()
}()
go func() { <-ctx.Done(); x.fail("отменено") }()
args.Relay, args.Chunk = x.id, xferChunk
inArgs := args
args.Conn, inArgs.Conn = q.FromConn, q.ToConn
var in proto.TaskResult
inDone := make(chan struct{})
go func() {
defer close(inDone)
res, err := s.waitTask(ctx, q.To, proto.TaskFSShipIn, inArgs, xferTimeout)
if err != nil {
res = proto.TaskResult{Error: err.Error()}
}
in = oldAgent(res)
if !in.OK {
x.fail(in.Error)
}
}()
out, err := s.waitTask(ctx, from, proto.TaskFSShipOut, args, xferTimeout)
if err != nil {
out = proto.TaskResult{Error: err.Error()}
}
out = oldAgent(out)
if !out.OK {
x.fail(out.Error)
}
<-inDone
if !in.OK {
return proto.TaskResult{Error: "на той машине: " + in.Error}
}
if !out.OK {
return out
}
if q.Move {
for _, p := range append([]string{q.Src}, q.More...) {
task, del := proto.TaskFSDelTree, any(proto.FSPathArgs{Path: p})
if q.FromConn != "" {
raw, _ := json.Marshal(proto.FSPathArgs{Path: p})
task, del = proto.TaskSSHFSOp, proto.SSHFSArgs{Conn: q.FromConn, Op: "deltree", Args: raw}
}
res, err := s.waitTask(ctx, from, task, del, fsTaskTimeout)
if err == nil && !res.OK {
err = errText(res.Error)
}
if err != nil {
return proto.TaskResult{Error: "отправлено, но здесь не удалилось: " + err.Error()}
}
}
}
// The receiver's count: what it put down and what it kept as it was.
return in
}
type errText string
func (e errText) Error() string { return string(e) }
// GET /api/panel/fs/job/{jid}?v=N - the job's state once it differs from
// version N, or after a while anyway.
func (s *Server) handleShipJob(w http.ResponseWriter, r *http.Request) {
j := s.ship.job(r.PathValue("jid"))
if j == nil {
writeJSON(w, map[string]any{"done": true, "result": proto.TaskResult{Error: "задание не найдено"}})
return
}
seen, _ := strconv.Atoi(r.URL.Query().Get("v"))
t := time.NewTimer(25 * time.Second)
defer t.Stop()
for {
j.mu.Lock()
if j.ver != seen || j.done {
out := map[string]any{"v": j.ver, "progress": j.prog, "done": j.done}
if j.done {
out["result"] = j.res
}
j.mu.Unlock()
writeJSON(w, out)
return
}
ch := j.changed
j.mu.Unlock()
select {
case <-ch:
case <-t.C:
seen = -1 // answer with what there is
case <-r.Context().Done():
return
}
}
}
// POST /api/panel/fs/job/{jid}/cancel
func (s *Server) handleShipCancel(w http.ResponseWriter, r *http.Request) {
if j := s.ship.job(r.PathValue("jid")); j != nil {
j.update(func() { j.stopped = true })
j.cancel()
}
w.WriteHeader(http.StatusNoContent)
}
// POST /api/agent/fs/job/{jid}/progress - the sender's progress. 410: the
// job is over or cancelled, stop.
func (s *Server) handleShipProgress(w http.ResponseWriter, r *http.Request) {
if s.authenticateAgent(r) == nil {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
j := s.ship.job(r.PathValue("jid"))
var p proto.ShipProgress
if j == nil || json.NewDecoder(http.MaxBytesReader(w, r.Body, 4096)).Decode(&p) != nil {
http.Error(w, "gone", http.StatusGone)
return
}
stop := false
j.update(func() { j.prog, stop = p, j.stopped || j.done })
if stop {
http.Error(w, "gone", http.StatusGone)
return
}
w.WriteHeader(http.StatusNoContent)
}
// POST /api/agent/fs/relay/{rid}/{i} - piece i of the stream from the
// sender, X-Last on the last. 503: no room yet, again; 410: it is over.
func (s *Server) handleRelayPost(w http.ResponseWriter, r *http.Request) {
a := s.authenticateAgent(r)
if a == nil {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
x := s.ship.relay(r.PathValue("rid"))
i, err := strconv.Atoi(r.PathValue("i"))
if x == nil || x.from != a.ID || err != nil || i < 0 {
http.Error(w, "передача не найдена", http.StatusGone)
return
}
data, err := readPiece(w, r)
if err != nil {
http.Error(w, "кусок не дошёл", http.StatusBadRequest)
return
}
if !x.await(r.Context(), xferWait, func() bool { return x.err != "" || i < x.low+xferBuffer }) {
http.Error(w, "нет места, повторите", http.StatusServiceUnavailable)
return
}
defer x.mu.Unlock()
if x.err != "" {
http.Error(w, x.err, http.StatusGone)
return
}
if i >= x.low && x.chunks[i] == nil {
x.chunks[i] = data
if r.Header.Get("X-Last") != "" {
x.end = i
}
x.notify()
}
w.WriteHeader(http.StatusNoContent)
}
// GET /api/agent/fs/relay/{rid}/{i} - piece i for the receiver, X-Last on
// the last. Asking for i says every piece below it is written, so those are
// let go. 503: not here yet, ask again; 410: it is over.
func (s *Server) handleRelayGet(w http.ResponseWriter, r *http.Request) {
a := s.authenticateAgent(r)
if a == nil {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
x := s.ship.relay(r.PathValue("rid"))
i, err := strconv.Atoi(r.PathValue("i"))
if x == nil || x.to != a.ID || err != nil || i < 0 {
http.Error(w, "передача не найдена", http.StatusGone)
return
}
x.mu.Lock()
if i > x.low {
for k := x.low; k < i; k++ {
delete(x.chunks, k)
}
x.low = i
x.notify()
}
x.mu.Unlock()
if !x.await(r.Context(), xferWait, func() bool { return x.err != "" || x.chunks[i] != nil }) {
http.Error(w, "кусок ещё не пришёл", http.StatusServiceUnavailable)
return
}
if x.err != "" {
msg := x.err
x.mu.Unlock()
http.Error(w, msg, http.StatusGone)
return
}
data, last := x.chunks[i], i == x.end
x.mu.Unlock()
w.Header().Set("Content-Type", "application/octet-stream")
w.Header().Set("Content-Length", strconv.Itoa(len(data)))
if last {
w.Header().Set("X-Last", "1")
}
_, _ = w.Write(data)
}
// shipsSeveral: both agents take several things in one stream (0.2.0 on).
func (s *Server) shipsSeveral(from string, q shipReq) bool {
if q.SSH != nil {
return false
}
for _, id := range []string{from, q.To} {
a, _ := s.db.GetAgent(id)
if a == nil || !versionAtLeast(a.Version, "0.2.0") {
return false
}
}
return true
}
// versionAtLeast: version v (dotted numbers) is min or newer; anything else
// is taken for older.
func versionAtLeast(v, min string) bool {
a, b := strings.Split(v, "."), strings.Split(min, ".")
for i := range b {
var x, y int
if i < len(a) {
n, err := strconv.Atoi(a[i])
if err != nil {
return false
}
x = n
}
y, _ = strconv.Atoi(b[i])
if x != y {
return x > y
}
}
return true
}
+109
View File
@@ -0,0 +1,109 @@
package server
import (
"encoding/json"
"io"
"net/http"
"strings"
"github.com/x-grk/longpolling-remote-access/internal/proto"
)
// Files on an SSH host through a machine, for the panel's file panes: the
// machine's agent keeps the SSH connection (open gives its id) and runs the
// operations there. Downloads and uploads go in the same pieces as a
// machine's own files.
// POST /api/panel/agents/{id}/sshfs/open - {"ssh": {...}} → {conn, home}.
func (s *Server) handleSSHFSOpen(w http.ResponseWriter, r *http.Request) {
agentID := r.PathValue("id")
var in struct {
SSH *proto.SSHTarget `json:"ssh"`
}
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 64<<10)).Decode(&in); err != nil || in.SSH == nil {
writeJSON(w, proto.TaskResult{Error: "не указан хост"})
return
}
if err := s.fillSaved(s.currentAdmin(r).ID, in.SSH); err != nil {
writeJSON(w, proto.TaskResult{Error: err.Error()})
return
}
t := in.SSH
t.Host, t.User = strings.TrimSpace(t.Host), strings.TrimSpace(t.User)
if t.Port == 0 {
t.Port = 22
}
if t.Host == "" || t.User == "" || t.Port < 1 || t.Port > 65535 {
writeJSON(w, proto.TaskResult{Error: "нужны хост, порт и пользователь"})
return
}
if !s.fsAgent(w, agentID, true) {
return
}
res, err := s.waitTask(r.Context(), agentID, proto.TaskSSHFSOpen, proto.SSHFSArgs{SSH: t}, fsTaskTimeout)
if err != nil {
res = proto.TaskResult{Error: err.Error()}
}
if unknownOp(res) {
res = proto.TaskResult{Error: "агент не умеет файлы по SSH - обновите его"}
}
if res.OK {
if admin := s.currentAdmin(r); admin != nil {
s.db.Audit(admin.ID, agentID, "sshfs.open:"+t.User+"@"+t.Host)
}
}
writeJSON(w, res)
}
// sshfsChanges are the operations that change files, for the audit log.
var sshfsChanges = map[string]bool{"write": true, "mkdir": true, "delete": true, "deltree": true, "move": true, "copy": true, "chmod": true, "zip": true, "unzip": true}
// POST /api/panel/agents/{id}/sshfs/{conn}/{op} - an operation on the host;
// the body is in the shape of the machine's own fs.{op}. "close" drops the
// connection.
func (s *Server) handleSSHFSOp(w http.ResponseWriter, r *http.Request) {
agentID, conn, op := r.PathValue("id"), r.PathValue("conn"), r.PathValue("op")
body, err := io.ReadAll(http.MaxBytesReader(w, r.Body, 64<<20))
if err != nil {
http.Error(w, "bad request", http.StatusBadRequest)
return
}
if !s.fsAgent(w, agentID, true) {
return
}
task, args := proto.TaskSSHFSOp, proto.SSHFSArgs{Conn: conn, Op: op, Args: json.RawMessage(body)}
if op == "close" {
task, args = proto.TaskSSHFSClose, proto.SSHFSArgs{Conn: conn}
}
if len(args.Args) == 0 {
args.Args = json.RawMessage("{}")
}
if sshfsChanges[op] {
var d struct{ Path, Src, Dst string }
_ = json.Unmarshal(body, &d)
if admin := s.currentAdmin(r); admin != nil {
s.db.Audit(admin.ID, agentID, "sshfs."+op+":"+strings.TrimSpace(d.Path+" "+d.Src+" "+d.Dst))
}
}
res, err := s.waitTask(r.Context(), agentID, task, args, fsTaskTimeout)
if err != nil {
res = proto.TaskResult{Error: err.Error()}
}
if unknownOp(res) && op != "close" {
res = proto.TaskResult{Error: "агент не умеет файлы по SSH - обновите его"}
}
writeJSON(w, res)
}
// GET /api/panel/agents/{id}/sshfs/{conn}/download?path=...[&inline=1]
func (s *Server) handleSSHFSDownload(w http.ResponseWriter, r *http.Request) {
s.download(w, r, proto.TaskSSHFSSend, r.PathValue("conn"))
}
// POST /api/panel/agents/{id}/sshfs/{conn}/upload?path=...&size=N
func (s *Server) handleSSHFSUpload(w http.ResponseWriter, r *http.Request) {
if admin := s.currentAdmin(r); admin != nil {
s.db.Audit(admin.ID, r.PathValue("id"), "sshfs.upload:"+r.URL.Query().Get("path"))
}
s.uploadStart(w, r, proto.TaskSSHFSPut, r.PathValue("conn"))
}
+211
View File
@@ -0,0 +1,211 @@
package server
import (
"crypto/aes"
"crypto/cipher"
"crypto/rand"
"encoding/base64"
"encoding/hex"
"encoding/json"
"errors"
"net/http"
"os"
"strconv"
"strings"
"sync"
"github.com/x-grk/longpolling-remote-access/internal/proto"
)
// Saved SSH hosts: each user keeps their own list of hosts with the way in -
// password or key - so the SSH dialog connects with one choice. The secret is
// sealed with AES-256-GCM under the server's key, which lives apart from the
// database (LPRA_SECRET_KEY, or a file beside the database made on first
// use), and bound to its owner: a row read out of the database, or moved to
// another user, opens to nobody.
type savedHost struct {
ID int64 `json:"id"`
Label string `json:"label"`
Host string `json:"host"`
Port int `json:"port"`
User string `json:"user"`
Auth string `json:"auth"` // password, key or agent
}
type sshSecret struct {
Password string `json:"password,omitempty"`
Key string `json:"key,omitempty"`
Passphrase string `json:"passphrase,omitempty"`
}
var (
secretKeyOnce sync.Once
secretKey []byte
secretKeyErr error
)
// sealKey is the server's key for saved secrets.
func (s *Server) sealKey() ([]byte, error) {
secretKeyOnce.Do(func() {
if v := strings.TrimSpace(os.Getenv("LPRA_SECRET_KEY")); v != "" {
k, err := hex.DecodeString(v)
if err != nil || len(k) != 32 {
k, err = base64.StdEncoding.DecodeString(v)
}
if err != nil || len(k) != 32 {
secretKeyErr = errors.New("LPRA_SECRET_KEY: нужны 32 байта (hex или base64)")
return
}
secretKey = k
return
}
path := s.db.path + ".key"
if b, err := os.ReadFile(path); err == nil {
k, err := hex.DecodeString(strings.TrimSpace(string(b)))
if err != nil || len(k) != 32 {
secretKeyErr = errors.New(path + ": испорчен ключ")
return
}
secretKey = k
return
}
k := make([]byte, 32)
_, _ = rand.Read(k)
if err := os.WriteFile(path, []byte(hex.EncodeToString(k)+"\n"), 0o600); err != nil {
secretKeyErr = err
return
}
secretKey = k
})
return secretKey, secretKeyErr
}
func (s *Server) seal(owner int64, plain []byte) (string, error) {
k, err := s.sealKey()
if err != nil {
return "", err
}
blk, _ := aes.NewCipher(k)
g, _ := cipher.NewGCM(blk)
nonce := make([]byte, g.NonceSize())
_, _ = rand.Read(nonce)
out := g.Seal(nonce, nonce, plain, []byte("ssh:"+strconv.FormatInt(owner, 10)))
return base64.StdEncoding.EncodeToString(out), nil
}
func (s *Server) open(owner int64, sealed string) ([]byte, error) {
k, err := s.sealKey()
if err != nil {
return nil, err
}
b, err := base64.StdEncoding.DecodeString(sealed)
blk, _ := aes.NewCipher(k)
g, _ := cipher.NewGCM(blk)
if err != nil || len(b) < g.NonceSize() {
return nil, errors.New("испорчена запись")
}
return g.Open(nil, b[:g.NonceSize()], b[g.NonceSize():], []byte("ssh:"+strconv.FormatInt(owner, 10)))
}
func (s *Server) saveSSHHost(owner int64, h savedHost, sec sshSecret) (int64, error) {
plain, _ := json.Marshal(sec)
sealed, err := s.seal(owner, plain)
if err != nil {
return 0, err
}
return s.db.sql.insert(`INSERT INTO ssh_hosts (admin_id, label, host, port, login, auth, secret) VALUES (?, ?, ?, ?, ?, ?, ?)`,
owner, h.Label, h.Host, h.Port, h.User, h.Auth, sealed)
}
func (s *Server) listSSHHosts(owner int64) []savedHost {
rows, err := s.db.sql.Query(`SELECT id, label, host, port, login, auth FROM ssh_hosts WHERE admin_id = ? ORDER BY label, host`, owner)
list := []savedHost{}
if err != nil {
return list
}
defer rows.Close()
for rows.Next() {
var h savedHost
if rows.Scan(&h.ID, &h.Label, &h.Host, &h.Port, &h.User, &h.Auth) == nil {
list = append(list, h)
}
}
return list
}
// fillSaved puts the saved host t.Saved of owner - address and way in - into
// t, which then goes on as if typed in.
func (s *Server) fillSaved(owner int64, t *proto.SSHTarget) error {
if t == nil || t.Saved == 0 {
return nil
}
var h savedHost
var sealed string
err := s.db.sql.QueryRow(`SELECT host, port, login, auth, secret FROM ssh_hosts WHERE id = ? AND admin_id = ?`, t.Saved, owner).
Scan(&h.Host, &h.Port, &h.User, &h.Auth, &sealed)
if err != nil {
return errors.New("сохранённый хост не найден")
}
plain, err := s.open(owner, sealed)
if err != nil {
return errors.New("сохранённый хост не расшифровался")
}
var sec sshSecret
_ = json.Unmarshal(plain, &sec)
*t = proto.SSHTarget{Host: h.Host, Port: h.Port, User: h.User, Password: sec.Password, Key: sec.Key, Passphrase: sec.Passphrase}
return nil
}
// GET /api/panel/ssh/hosts - the user's saved hosts, without secrets.
func (s *Server) handleSSHHosts(w http.ResponseWriter, r *http.Request) {
writeJSON(w, s.listSSHHosts(s.currentAdmin(r).ID))
}
// POST /api/panel/ssh/hosts - save a host: {label, host, port, user, auth,
// password | key, passphrase}.
func (s *Server) handleSSHHostAdd(w http.ResponseWriter, r *http.Request) {
var in struct {
savedHost
sshSecret
}
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 64<<10)).Decode(&in); err != nil {
writeJSON(w, map[string]any{"ok": false, "error": "плохой запрос"})
return
}
h := in.savedHost
h.Host, h.User, h.Label = strings.TrimSpace(h.Host), strings.TrimSpace(h.User), strings.TrimSpace(h.Label)
if h.Port == 0 {
h.Port = 22
}
if h.Host == "" || h.User == "" || h.Port < 1 || h.Port > 65535 || strings.ContainsAny(h.Host+h.User, " \t\r\n/") {
writeJSON(w, map[string]any{"ok": false, "error": "нужны хост, порт и пользователь"})
return
}
if h.Label == "" {
h.Label = h.User + "@" + h.Host
}
sec := in.sshSecret
switch h.Auth {
case "key":
sec.Password = ""
case "agent":
sec = sshSecret{}
default:
h.Auth = "password"
sec.Key, sec.Passphrase = "", ""
}
id, err := s.saveSSHHost(s.currentAdmin(r).ID, h, sec)
if err != nil {
writeJSON(w, map[string]any{"ok": false, "error": err.Error()})
return
}
writeJSON(w, map[string]any{"ok": true, "id": id})
}
// POST /api/panel/ssh/hosts/{hid}/delete
func (s *Server) handleSSHHostDelete(w http.ResponseWriter, r *http.Request) {
id, _ := strconv.ParseInt(r.PathValue("hid"), 10, 64)
_, _ = s.db.sql.Exec(`DELETE FROM ssh_hosts WHERE id = ? AND admin_id = ?`, id, s.currentAdmin(r).ID)
writeJSON(w, map[string]any{"ok": true})
}
+54
View File
@@ -0,0 +1,54 @@
package server
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/x-grk/longpolling-remote-access/internal/proto"
)
// A saved host's secret is stored encrypted, and only its owner gets it back.
func TestSavedSSHHosts(t *testing.T) {
s, _ := newTestServer(t)
a, _ := s.db.AdminByUsername("adm")
id, err := s.saveSSHHost(a.ID, savedHost{Label: "web", Host: "h", Port: 22, User: "root", Auth: "password"}, sshSecret{Password: "s3cret-pass"})
if err != nil {
t.Fatal(err)
}
var raw string
_ = s.db.sql.QueryRow(`SELECT secret FROM ssh_hosts WHERE id = ?`, id).Scan(&raw)
if raw == "" || strings.Contains(raw, "s3cret-pass") {
t.Fatalf("secret stored as %q", raw)
}
got := &proto.SSHTarget{Saved: id}
if err := s.fillSaved(a.ID, got); err != nil || got.Password != "s3cret-pass" || got.Host != "h" || got.User != "root" || got.Saved != 0 {
t.Fatalf("own: %+v %v", got, err)
}
if err := s.fillSaved(a.ID+1, &proto.SSHTarget{Saved: id}); err == nil {
t.Fatal("another user got the host")
}
if list := s.listSSHHosts(a.ID); len(list) != 1 || list[0].Label != "web" {
t.Fatalf("list: %+v", list)
}
if len(s.listSSHHosts(a.ID+1)) != 0 {
t.Fatal("another user sees the host")
}
}
// The routes: a saved host shows without its secret.
func TestSavedSSHHostsRoute(t *testing.T) {
s, c := newTestServer(t)
req := httptest.NewRequest(http.MethodPost, "/api/panel/ssh/hosts", strings.NewReader(`{"label":"db","host":"h2","user":"u","auth":"password","password":"pw-xyz"}`))
req.AddCookie(c)
w := httptest.NewRecorder()
s.Handler().ServeHTTP(w, req)
if !strings.Contains(w.Body.String(), `"ok":true`) {
t.Fatalf("save: %s", w.Body.String())
}
w = get(t, s, c, "/api/panel/ssh/hosts")
if !strings.Contains(w.Body.String(), `"h2"`) || strings.Contains(w.Body.String(), "pw-xyz") {
t.Fatalf("list: %s", w.Body.String())
}
}
+23
View File
@@ -0,0 +1,23 @@
package server
import (
"net/http"
"net/http/httptest"
"testing"
"testing/fstest"
)
func TestStaticHidesTests(t *testing.T) {
fsys := fstest.MapFS{
"wsutil.js": {Data: []byte("x")},
"wsutil.test.js": {Data: []byte("y")},
}
h := staticHandler(fsys, map[string]string{})
for path, want := range map[string]int{"/wsutil.js": 200, "/wsutil.test.js": 404} {
w := httptest.NewRecorder()
h.ServeHTTP(w, httptest.NewRequest(http.MethodGet, path, nil))
if w.Code != want {
t.Errorf("%s: got %d, want %d", path, w.Code, want)
}
}
}
+34
View File
@@ -0,0 +1,34 @@
package server
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
)
// A terminal opened with a folder asks the agent to start there; a folder
// that is not an absolute path is left out (the shell starts at home).
func TestPanelOpenDir(t *testing.T) {
s, cookie := newTestServer(t)
for body, want := range map[string]string{
`{"dir":"/srv/my app"}`: "/srv/my app",
`{"dir":"relative"}`: "",
``: "",
} {
req := httptest.NewRequest(http.MethodPost, "/api/panel/term/x1/open", strings.NewReader(body))
req.AddCookie(cookie)
w := httptest.NewRecorder()
s.Handler().ServeHTTP(w, req)
if w.Code != 200 {
t.Fatalf("%s: code %d %s", body, w.Code, w.Body)
}
s.hub.mu.Lock()
jobs := s.hub.jobs["x1"]
s.hub.jobs["x1"] = nil
s.hub.mu.Unlock()
if len(jobs) != 1 || jobs[0].Dir != want {
t.Fatalf("%s: jobs %+v, want dir %q", body, jobs, want)
}
}
}
+732
View File
@@ -0,0 +1,732 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=3.0, user-scalable=1;">
<meta name="referrer" content="origin">
<meta name="HandheldFriendly" content="true" />
<meta name="robots" content="noindex, nofollow" />
<title>golang context cancel at DuckDuckGo</title>
<link title="DuckDuckGo (Lite)" type="application/opensearchdescription+xml" rel="search" href="//duckduckgo.com/opensearch_lite_v2.xml">
<link href="//duckduckgo.com/favicon.ico" rel="shortcut icon" />
<link rel="icon" href="//duckduckgo.com/favicon.ico" type="image/x-icon" />
<link id="icon60" rel="apple-touch-icon" href="//duckduckgo.com/assets/icons/meta/DDG-iOS-icon_60x60.png?v=2"/>
<link id="icon76" rel="apple-touch-icon" sizes="76x76" href="//duckduckgo.com/assets/icons/meta/DDG-iOS-icon_76x76.png?v=2"/>
<link id="icon120" rel="apple-touch-icon" sizes="120x120" href="//duckduckgo.com/assets/icons/meta/DDG-iOS-icon_120x120.png?v=2"/>
<link id="icon152" rel="apple-touch-icon" sizes="152x152" href="//duckduckgo.com/assets/icons/meta/DDG-iOS-icon_152x152.png?v=2"/>
<link rel="image_src" href="//duckduckgo.com/assets/icons/meta/DDG-icon_256x256.png">
<link rel="stylesheet" media="handheld, all" href="//duckduckgo.com/dist/lr.48ddfe4eadf6a534e93f.css" type="text/css"/>
</head>
<body>
<p class='extra'>&nbsp;</p>
<div class="header">
DuckDuckGo
</div>
<p class='extra'>&nbsp;</p>
<form action="/lite/" method="post">
<input class='query' type="text" size="40" name="q" value="golang context cancel">
<input class='submit' type="submit" value="Search">
<div class="filters">
<select class="submit" name="kl">
<option value="" >All Regions</option>
<option value="ar-es" >Argentina</option>
<option value="au-en" >Australia</option>
<option value="at-de" >Austria</option>
<option value="be-fr" >Belgium (fr)</option>
<option value="be-nl" >Belgium (nl)</option>
<option value="br-pt" >Brazil</option>
<option value="bg-bg" >Bulgaria</option>
<option value="ca-en" >Canada (en)</option>
<option value="ca-fr" >Canada (fr)</option>
<option value="ct-ca" >Catalonia</option>
<option value="cl-es" >Chile</option>
<option value="cn-zh" >China</option>
<option value="co-es" >Colombia</option>
<option value="hr-hr" >Croatia</option>
<option value="cz-cs" >Czech Republic</option>
<option value="dk-da" >Denmark</option>
<option value="ee-et" >Estonia</option>
<option value="fi-fi" >Finland</option>
<option value="fr-fr" >France</option>
<option value="de-de" >Germany</option>
<option value="gr-el" >Greece</option>
<option value="hk-tzh" >Hong Kong</option>
<option value="hu-hu" >Hungary</option>
<option value="is-is" >Iceland</option>
<option value="in-en" >India (en)</option>
<option value="id-en" >Indonesia (en)</option>
<option value="ie-en" >Ireland</option>
<option value="il-en" >Israel (en)</option>
<option value="it-it" >Italy</option>
<option value="jp-jp" >Japan</option>
<option value="kr-kr" >Korea</option>
<option value="lv-lv" >Latvia</option>
<option value="lt-lt" >Lithuania</option>
<option value="my-en" >Malaysia (en)</option>
<option value="mx-es" >Mexico</option>
<option value="nl-nl" >Netherlands</option>
<option value="nz-en" >New Zealand</option>
<option value="no-no" >Norway</option>
<option value="pk-en" >Pakistan (en)</option>
<option value="pe-es" >Peru</option>
<option value="ph-en" >Philippines (en)</option>
<option value="pl-pl" >Poland</option>
<option value="pt-pt" >Portugal</option>
<option value="ro-ro" >Romania</option>
<option value="ru-ru" >Russia</option>
<option value="xa-ar" >Saudi Arabia</option>
<option value="sg-en" >Singapore</option>
<option value="sk-sk" >Slovakia</option>
<option value="sl-sl" >Slovenia</option>
<option value="za-en" >South Africa</option>
<option value="es-ca" >Spain (ca)</option>
<option value="es-es" >Spain (es)</option>
<option value="se-sv" >Sweden</option>
<option value="ch-de" >Switzerland (de)</option>
<option value="ch-fr" >Switzerland (fr)</option>
<option value="tw-tzh" >Taiwan</option>
<option value="th-en" >Thailand (en)</option>
<option value="tr-tr" >Turkey</option>
<option value="us-en" >US (English)</option>
<option value="us-es" >US (Spanish)</option>
<option value="ua-uk" >Ukraine</option>
<option value="uk-en" >United Kingdom</option>
<option value="vn-en" >Vietnam (en)</option>
</select>
<select class="submit" name="df">
<option value="" selected>Any Time</option>
<option value="d" >Past Day</option>
<option value="w" >Past Week</option>
<option value="m" >Past Month</option>
<option value="y" >Past Year</option>
</select>
</div>
</form>
<p class="extra">&nbsp;</p>
<table border="0">
<tr>
<td>
</td>
<td>
<!-- Next Page Button Sub-template -->
<form class="next_form" action="/lite/" method="post">
<input type="submit" class='navbutton' value="Next Page &gt;">
<input type="hidden" name="q" value="golang context cancel">
<input type="hidden" name="s" value="10">
<input type="hidden" name="nextParams" value="">
<input type="hidden" name="v" value="l">
<input type="hidden" name="o" value="json">
<input type="hidden" name="dc" value="11">
<input type="hidden" name="api" value="d.js">
<input type="hidden" name="vqd" value="4-288446547072343849888557955747865435943">
&nbsp;&nbsp;&nbsp;&nbsp;
<input name="kl" value="wt-wt" type="hidden">
</form>
</td>
</tr>
</table>
<p class='extra'>&nbsp;</p>
<table border="0">
</table>
<table border="0">
<!-- Web results are present -->
<tr>
<td valign="top">
1.&nbsp;
</td>
<td>
<a rel="nofollow" href="//duckduckgo.com/l/?uddg=https%3A%2F%2Fgo.dev%2Fdoc%2Fdatabase%2Fcancel%2Doperations&amp;rut=114b1aa13c9158c8c2b077ccd5f890f8a940277c1fc938e7bdcaad636c444a06" class='result-link'>Canceling in-progress operations - The Go Programming Language</a>
</td>
</tr>
<!-- Only show abstract separately if there's a click URL (not EOF) -->
<tr>
<td>&nbsp;&nbsp;&nbsp;</td>
<td class='result-snippet'>
Learn how to use <b>context</b>.<b>Context</b> to manage and <b>cancel</b> database operations in Go. See examples of how to set a timeout, deadline, and propagate cancellation requests with <b>context</b>.
</td>
</tr>
<tr>
<td>&nbsp;&nbsp;&nbsp;</td>
<td>
<span class='link-text'>go.dev/doc/database/cancel-operations</span>
</td>
</tr>
<tr>
<td>&nbsp;</td>
<td>&nbsp;</td>
</tr>
<tr>
<td valign="top">
2.&nbsp;
</td>
<td>
<a rel="nofollow" href="//duckduckgo.com/l/?uddg=https%3A%2F%2Fgolangtutorial.com%2Fgolang%2Dcontext%2F&amp;rut=c8e3cbb5601e47ba9c55f0ad6b52790e09acc1c0d3caf33c676b68ee65a1f419" class='result-link'>Go Context Tutorial: Cancellation and Deadlines</a>
</td>
</tr>
<!-- Only show abstract separately if there's a click URL (not EOF) -->
<tr>
<td>&nbsp;&nbsp;&nbsp;</td>
<td class='result-snippet'>
Learn Go <b>context</b> with practical cancellation, timeout, HTTP request, and goroutine examples.
</td>
</tr>
<tr>
<td>&nbsp;&nbsp;&nbsp;</td>
<td>
<span class='link-text'>golangtutorial.com/golang-context/</span>
&nbsp;&nbsp;&nbsp;
<span class='timestamp'>2026-07-29T00:00:00.0000000</span>
</td>
</tr>
<tr>
<td>&nbsp;</td>
<td>&nbsp;</td>
</tr>
<tr>
<td valign="top">
3.&nbsp;
</td>
<td>
<a rel="nofollow" href="//duckduckgo.com/l/?uddg=https%3A%2F%2Fstackoverflow.com%2Fquestions%2F52799280%2Fcontext%2Dconfusion%2Dregarding%2Dcancellation&amp;rut=0fa1c305fa28cede62f85c49d0501954cac2eba160d5c48f155a1008367ae78c" class='result-link'>go - Context confusion regarding cancellation - Stack Overflow</a>
</td>
</tr>
<!-- Only show abstract separately if there's a click URL (not EOF) -->
<tr>
<td>&nbsp;&nbsp;&nbsp;</td>
<td class='result-snippet'>
Additional info #1: calling <b>cancel</b> () right after <b>context</b> initialized As per <b>context</b> documentation regarding the <b>cancel</b> () function: Canceling this <b>context</b> releases resources associated with it, so code should call <b>cancel</b> as soon as the operations running in this <b>Context</b> complete. It&#x27;s good to always call <b>cancel</b> () function right after the <b>context</b> declaration. doesn&#x27;t matter whether it&#x27;s also ...
</td>
</tr>
<tr>
<td>&nbsp;&nbsp;&nbsp;</td>
<td>
<span class='link-text'>stackoverflow.com/questions/52799280/context-confusion-regarding-cancellation</span>
</td>
</tr>
<tr>
<td>&nbsp;</td>
<td>&nbsp;</td>
</tr>
<tr>
<td valign="top">
4.&nbsp;
</td>
<td>
<a rel="nofollow" href="//duckduckgo.com/l/?uddg=https%3A%2F%2Fpkg.go.dev%2Fcontext&amp;rut=925dd34b1fc94c2b0cb26f6e920e10aa3131c3cfc14163fd7c01b17554611b77" class='result-link'>context package - context - Go Packages</a>
</td>
</tr>
<!-- Only show abstract separately if there's a click URL (not EOF) -->
<tr>
<td>&nbsp;&nbsp;&nbsp;</td>
<td class='result-snippet'>
Package <b>context</b> defines the <b>Context</b> type, which carries deadlines, cancellation signals, and other request-scoped values across API boundaries and between processes.
</td>
</tr>
<tr>
<td>&nbsp;&nbsp;&nbsp;</td>
<td>
<span class='link-text'>pkg.go.dev/context</span>
&nbsp;&nbsp;&nbsp;
<span class='timestamp'>2026-10-08T00:00:00.0000000</span>
</td>
</tr>
<tr>
<td>&nbsp;</td>
<td>&nbsp;</td>
</tr>
<tr>
<td valign="top">
5.&nbsp;
</td>
<td>
<a rel="nofollow" href="//duckduckgo.com/l/?uddg=https%3A%2F%2Fstackoverflow.com%2Fquestions%2F70042646%2Fbest%2Dpractices%2Don%2Dgo%2Dcontext%2Dcancelation%2Dfunctions&amp;rut=f3a199db4b71e24ff5a3b0d4cbf7241ee53a03312bc2e2bcadf6d9cb20aaa474" class='result-link'>Best practices on go context cancelation functions</a>
</td>
</tr>
<!-- Only show abstract separately if there's a click URL (not EOF) -->
<tr>
<td>&nbsp;&nbsp;&nbsp;</td>
<td class='result-snippet'>
So, is passing this cancellation function actually an issue? Are there any best practices related to the use of the <b>context</b> package and their <b>cancel</b> function?
</td>
</tr>
<tr>
<td>&nbsp;&nbsp;&nbsp;</td>
<td>
<span class='link-text'>stackoverflow.com/questions/70042646/best-practices-on-go-context-cancelation-functions</span>
</td>
</tr>
<tr>
<td>&nbsp;</td>
<td>&nbsp;</td>
</tr>
<tr>
<td valign="top">
6.&nbsp;
</td>
<td>
<a rel="nofollow" href="//duckduckgo.com/l/?uddg=https%3A%2F%2Fgolangbot.com%2Fcontext%2Dtimeout%2Dcancellation%2F&amp;rut=dc6314fa589ab9eac3160a9b8df6a7ca4604e480917f0b99cf558d081c78ab02" class='result-link'>Golang Context - Cancellation, Timeout and Propagation | golangbot.com</a>
</td>
</tr>
<!-- Only show abstract separately if there's a click URL (not EOF) -->
<tr>
<td>&nbsp;&nbsp;&nbsp;</td>
<td class='result-snippet'>
Learn how to <b>cancel</b> long running tasks in Go using <b>context</b> cancellation and timeout. This tutorial also teaches how to terminate outstanding web requests which were cancelled by the client.
</td>
</tr>
<tr>
<td>&nbsp;&nbsp;&nbsp;</td>
<td>
<span class='link-text'>golangbot.com/context-timeout-cancellation/</span>
&nbsp;&nbsp;&nbsp;
<span class='timestamp'>2026-07-08T00:00:00.0000000</span>
</td>
</tr>
<tr>
<td>&nbsp;</td>
<td>&nbsp;</td>
</tr>
<tr>
<td valign="top">
7.&nbsp;
</td>
<td>
<a rel="nofollow" href="//duckduckgo.com/l/?uddg=https%3A%2F%2Fgo.dev%2Fblog%2Fcontext&amp;rut=6d324e0f7c589513830578b05d2a0a5ccedf509c8d4181ac590b3a229c8b9057" class='result-link'>Go Concurrency Patterns: Context - The Go Programming Language</a>
</td>
</tr>
<!-- Only show abstract separately if there's a click URL (not EOF) -->
<tr>
<td>&nbsp;&nbsp;&nbsp;</td>
<td class='result-snippet'>
Learn how to use the <b>context</b> package to pass request-scoped values, cancellation signals, and deadlines across API boundaries in Go servers. See a complete example of an HTTP server that handles Google Web Search requests with <b>context</b>.
</td>
</tr>
<tr>
<td>&nbsp;&nbsp;&nbsp;</td>
<td>
<span class='link-text'>go.dev/blog/context</span>
</td>
</tr>
<tr>
<td>&nbsp;</td>
<td>&nbsp;</td>
</tr>
<tr>
<td valign="top">
8.&nbsp;
</td>
<td>
<a rel="nofollow" href="//duckduckgo.com/l/?uddg=https%3A%2F%2Fwww.sohamkamani.com%2Fgolang%2Fcontext%2F&amp;rut=7274980ce55c2a5b88f9299c12e1228a7ecfffbb5fabbab8a837cbad990a1a87" class='result-link'>How to Use Context in Golang (Deadlines, Cancellation, and Passing Values)</a>
</td>
</tr>
<!-- Only show abstract separately if there's a click URL (not EOF) -->
<tr>
<td>&nbsp;&nbsp;&nbsp;</td>
<td class='result-snippet'>
In this post, we&#x27;ll learn about Go&#x27;s <b>context</b> package, and more specifically, how we can use it to improve our applications performance. The main purpose of using <b>Context</b> is to manage long-running processes (like an HTTP request, a database call, or a network call) in a way that doesn&#x27;t waste resources. If used correctly, the <b>context</b> package can help you: <b>Cancel</b> long running processes ...
</td>
</tr>
<tr>
<td>&nbsp;&nbsp;&nbsp;</td>
<td>
<span class='link-text'>www.sohamkamani.com/golang/context/</span>
</td>
</tr>
<tr>
<td>&nbsp;</td>
<td>&nbsp;</td>
</tr>
<tr>
<td valign="top">
9.&nbsp;
</td>
<td>
<a rel="nofollow" href="//duckduckgo.com/l/?uddg=https%3A%2F%2Fgolangdoc.github.io%2Fpkg%2F1.12%2Fcontext%2Findex.html&amp;rut=4baaf27a174aa7e8bbcc0cfe1985ad916fe3ff014eb45fc06dd68178fc84e261" class='result-link'>context - The Go Programming Language</a>
</td>
</tr>
<!-- Only show abstract separately if there's a click URL (not EOF) -->
<tr>
<td>&nbsp;&nbsp;&nbsp;</td>
<td class='result-snippet'>
The returned <b>context&#x27;s</b> Done channel is closed when the deadline expires, when the returned <b>cancel</b> function is called, or when the parent <b>context&#x27;s</b> Done channel is closed, whichever happens first.
</td>
</tr>
<tr>
<td>&nbsp;&nbsp;&nbsp;</td>
<td>
<span class='link-text'>golangdoc.github.io/pkg/1.12/context/index.html</span>
</td>
</tr>
<tr>
<td>&nbsp;</td>
<td>&nbsp;</td>
</tr>
<tr>
<td valign="top">
10.&nbsp;
</td>
<td>
<a rel="nofollow" href="//duckduckgo.com/l/?uddg=https%3A%2F%2Fwww.golinuxcloud.com%2Fgolang%2Dcontext%2F&amp;rut=efdde246db353602fc52ef95d8d09a146270aeb46114aa061f320d037adea574" class='result-link'>Golang context — WithCancel, WithTimeout, WithDeadline, Background</a>
</td>
</tr>
<!-- Only show abstract separately if there's a click URL (not EOF) -->
<tr>
<td>&nbsp;&nbsp;&nbsp;</td>
<td class='result-snippet'>
The <b>golang</b> <b>context</b> package ties cancellation, timeouts, and optional request values to a <b>Context</b> value you pass through your call stack. Use Background (or TODO temporarily) as the root, WithTimeout / WithDeadline for time bounds, WithCancel for explicit shutdown, and WithValue only for small, request-scoped metadata—always defer <b>cancel</b> ...
</td>
</tr>
<tr>
<td>&nbsp;&nbsp;&nbsp;</td>
<td>
<span class='link-text'>www.golinuxcloud.com/golang-context/</span>
</td>
</tr>
<tr>
<td>&nbsp;</td>
<td>&nbsp;</td>
</tr>
<tr>
<td colspan=2>
<!-- Next Page Button Sub-template -->
<form class="next_form" action="/lite/" method="post">
<input type="submit" class='navbutton' value="Next Page &gt;">
<input type="hidden" name="q" value="golang context cancel">
<input type="hidden" name="s" value="10">
<input type="hidden" name="nextParams" value="">
<input type="hidden" name="v" value="l">
<input type="hidden" name="o" value="json">
<input type="hidden" name="dc" value="11">
<input type="hidden" name="api" value="d.js">
<input type="hidden" name="vqd" value="4-288446547072343849888557955747865435943">
&nbsp;&nbsp;&nbsp;&nbsp;
<input name="kl" value="wt-wt" type="hidden">
</form>
</td>
</tr>
</table>
<p class='extra'>&nbsp;</p>
<form action="/lite/" method="post">
<input class='query' type="text" size="40" name="q" value="golang context cancel" >
<input class='submit' type="submit" value="Search">
<input name="kl" value="wt-wt" type="hidden">
</form>
<p class='extra'>&nbsp;</p>
<img src="//duckduckgo.com/t/sl_l"/>
<div id='end-spacer'>&nbsp;</div>
</body>
</html>
+20 -1
View File
@@ -39,6 +39,7 @@ func readLine(r io.Reader, max int) (string, error) {
type tunnelHub struct {
db *DB
mu sync.Mutex
ln net.Listener
agents map[string]*agentTunnel
}
@@ -57,12 +58,30 @@ type agentTunnel struct {
streams map[uint32]net.Conn
}
// listen runs the 8446 TCP tunnel acceptor until the listener closes.
// listen runs the TCP tunnel acceptor until the listener closes. Listening
// again moves the tunnel: the previous listener is closed and its agents are
// dropped, so they redial the new address.
func (h *tunnelHub) listen(addr string) error {
ln, err := net.Listen("tcp", addr)
if err != nil {
return err
}
h.mu.Lock()
old := h.ln
h.ln = ln
var stale []*agentTunnel
if old != nil {
for _, at := range h.agents {
stale = append(stale, at)
}
}
h.mu.Unlock()
if old != nil {
old.Close()
}
for _, at := range stale {
at.conn.Close()
}
log.Printf("tunnel: listening on %s (direct TCP forwarding)", addr)
go func() {
for {
+21 -9
View File
@@ -11,11 +11,11 @@ import (
"github.com/x-grk/longpolling-remote-access/internal/tunnel"
)
// udpRelay carries UDP forwards over the direct 8446 UDP channel. The agent
// udpRelay carries UDP forwards over the direct UDP channel. The agent
// registers its UDP return address; end-user datagrams arriving on a rule's
// host port are relayed to the agent and replies are sent back.
//
// Datagram framing on the 8446 control socket:
// Datagram framing on the tunnel control socket:
//
// registration (agent->server): "R <agentID> <sig>"
// data (both): 'D' | sid(4) | port(2) | payload
@@ -23,6 +23,7 @@ type udpRelay struct {
db *DB
mu sync.Mutex
ctrl *net.UDPConn
gcOnce sync.Once
agentAddr map[string]*net.UDPAddr // agentID -> last registered addr
sessions map[uint32]*udpSession
@@ -48,7 +49,8 @@ func newUDPRelay(db *DB) *udpRelay {
}
}
// listen opens the shared 8446 UDP control socket.
// listen opens the shared UDP control socket. Listening again moves it: the
// previous socket is closed and agents register anew on the new address.
func (u *udpRelay) listen(addr string) error {
ua, err := net.ResolveUDPAddr("udp", addr)
if err != nil {
@@ -58,17 +60,26 @@ func (u *udpRelay) listen(addr string) error {
if err != nil {
return err
}
u.mu.Lock()
old := u.ctrl
u.ctrl = c
if old != nil {
u.agentAddr = map[string]*net.UDPAddr{}
}
u.mu.Unlock()
if old != nil {
old.Close()
}
log.Printf("tunnel: listening on %s/udp (direct UDP forwarding)", addr)
go u.ctrlLoop()
go u.gc()
go u.ctrlLoop(c)
u.gcOnce.Do(func() { go u.gc() })
return nil
}
func (u *udpRelay) ctrlLoop() {
func (u *udpRelay) ctrlLoop(ctrl *net.UDPConn) {
buf := make([]byte, 65535)
for {
n, src, err := u.ctrl.ReadFromUDP(buf)
n, src, err := ctrl.ReadFromUDP(buf)
if err != nil {
return
}
@@ -171,6 +182,7 @@ func (u *udpRelay) rulePortLoop(rule ForwardRule, c *net.UDPConn) {
}
u.mu.Lock()
aAddr := u.agentAddr[rule.AgentID]
ctrl := u.ctrl
// Find existing session for this (rule, user).
var sess *udpSession
for _, s := range u.sessions {
@@ -190,7 +202,7 @@ func (u *udpRelay) rulePortLoop(rule ForwardRule, c *net.UDPConn) {
sess.last = time.Now()
u.mu.Unlock()
if aAddr == nil {
if aAddr == nil || ctrl == nil {
continue // agent has not registered its UDP endpoint yet
}
// 'D' | sid(4) | clientPort(2) | payload
@@ -199,7 +211,7 @@ func (u *udpRelay) rulePortLoop(rule ForwardRule, c *net.UDPConn) {
binary.BigEndian.PutUint32(out[1:5], sess.id)
binary.BigEndian.PutUint16(out[5:7], uint16(rule.ClientPort))
copy(out[7:], buf[:n])
_, _ = u.ctrl.WriteToUDP(out, aAddr)
_, _ = ctrl.WriteToUDP(out, aAddr)
}
}
+150 -32
View File
@@ -3,8 +3,11 @@ package server
import (
"fmt"
"log"
"net"
"os"
"os/exec"
"strings"
"sync"
)
// ufwManager integrates with the host firewall (ufw) when it is installed and
@@ -15,30 +18,52 @@ import (
// (e.g. running as root). When it cannot, integration is simply inactive.
type ufwManager struct {
db *DB
tunnelPort int
// tunnelPort returns the current direct-tunnel port (0 = tunnel disabled).
tunnelPort func() int
mu sync.Mutex
lastErr string // last failed ufw command, shown in Settings
}
func newUfwManager(db *DB, tunnelPort int) *ufwManager {
func newUfwManager(db *DB, tunnelPort func() int) *ufwManager {
return &ufwManager{db: db, tunnelPort: tunnelPort}
}
// ufwRun runs ufw with a fixed C locale: its output is translated, and we
// match on the English text.
func ufwRun(args ...string) (string, error) {
cmd := exec.Command("ufw", args...)
cmd.Env = append(os.Environ(), "LC_ALL=C", "LANG=C", "LANGUAGE=C")
out, err := cmd.CombinedOutput()
return strings.TrimSpace(string(out)), err
}
// status reports whether ufw is installed, active, and runnable by this process.
func (u *ufwManager) status() (installed, active, runnable bool) {
path, err := exec.LookPath("ufw")
if err != nil {
if _, err := exec.LookPath("ufw"); err != nil {
return false, false, false
}
installed = true
out, err := exec.Command(path, "status").CombinedOutput()
out, err := ufwRun("status")
if err != nil {
// Typically a permission error (not root) -> not runnable.
return true, false, false
}
runnable = true
active = strings.Contains(string(out), "Status: active")
active = strings.Contains(out, "Status: active")
return installed, active, runnable
}
// enabled reports whether the integration is switched on and ufw is there to
// drive. Unlike available it runs nothing, so pages can ask it freely.
func (u *ufwManager) enabled() bool {
if !u.db.UfwEnabled() {
return false
}
_, err := exec.LookPath("ufw")
return err == nil
}
// available reports whether we can and should drive ufw now.
func (u *ufwManager) available() bool {
if !u.db.UfwEnabled() {
@@ -48,30 +73,70 @@ func (u *ufwManager) available() bool {
return installed && active && runnable
}
// allow opens port/proto in ufw (if integration is available) and records it,
// bound to the given forward rule id (0 = shared/manual).
func (u *ufwManager) allow(port int, proto string, ruleID int64) {
if !u.available() {
return
func (u *ufwManager) setErr(msg string) {
u.mu.Lock()
u.lastErr = msg
u.mu.Unlock()
}
// LastError returns the last failed ufw command ("" when the last one worked).
func (u *ufwManager) LastError() string {
u.mu.Lock()
defer u.mu.Unlock()
return u.lastErr
}
// allow opens port/proto in ufw and records it, bound to the given forward
// rule id (0 = shared, ufwManual = added in Settings). It reports whether the
// panel now tracks the rule. The caller has checked available().
func (u *ufwManager) allow(port int, proto string, ruleID int64) bool {
if u.db.HasUfwRule(port, proto) {
return
return true
}
spec := fmt.Sprintf("%d/%s", port, proto)
if out, err := exec.Command("ufw", "allow", spec).CombinedOutput(); err != nil {
log.Printf("ufw allow %s: %v: %s", spec, err, strings.TrimSpace(string(out)))
return
out, err := ufwRun("allow", spec)
if err != nil {
log.Printf("ufw allow %s: %v: %s", spec, err, out)
u.setErr(fmt.Sprintf("ufw allow %s: %v: %s", spec, err, out))
return false
}
u.setErr("")
if strings.Contains(out, "Skipping") {
// The rule was there before the panel asked for it, so it is not ours
// to track (and later delete).
return false
}
_ = u.db.AddUfwRule(port, proto, ruleID)
log.Printf("ufw: allowed %s", spec)
return true
}
// allowManual opens a port the admin asked for in Settings. A port the panel
// already opened for a forward or the tunnel becomes the admin's, so it stays
// when they no longer need it. It returns the error key for the settings page
// ("" on success).
func (u *ufwManager) allowManual(port int, proto string) string {
if !u.available() {
return "ufw_off"
}
if !u.allow(port, proto, ufwManual) {
if u.LastError() != "" {
return "ufw_failed"
}
return "ufw_exists"
}
_ = u.db.SetUfwRuleManual(port, proto)
return ""
}
// revokeRule deletes one tracked ufw rule from ufw and the tracking table.
func (u *ufwManager) revokeRule(r UfwRule) {
spec := fmt.Sprintf("%d/%s", r.Port, r.Proto)
if out, err := exec.Command("ufw", "delete", "allow", spec).CombinedOutput(); err != nil {
log.Printf("ufw delete allow %s: %v: %s", spec, err, strings.TrimSpace(string(out)))
if out, err := ufwRun("delete", "allow", spec); err != nil {
log.Printf("ufw delete allow %s: %v: %s", spec, err, out)
u.setErr(fmt.Sprintf("ufw delete allow %s: %v: %s", spec, err, out))
} else {
u.setErr("")
log.Printf("ufw: revoked %s", spec)
}
_ = u.db.DeleteUfwRule(r.ID)
@@ -86,8 +151,8 @@ func (u *ufwManager) revoke(id int64) {
u.revokeRule(*r)
}
// revokeForRule removes the ufw rules bound to a specific forward rule. The
// shared tunnel port (rule_id 0) is left alone, since other forwards use it.
// revokeForRule removes the ufw rules bound to a specific forward rule, and
// the shared tunnel port once no forward needs it any more.
func (u *ufwManager) revokeForRule(ruleID int64) {
rules, err := u.db.ListUfwRulesByRule(ruleID)
if err != nil {
@@ -96,24 +161,77 @@ func (u *ufwManager) revokeForRule(ruleID int64) {
for _, r := range rules {
u.revokeRule(r)
}
u.pruneShared()
}
// pruneShared revokes shared (rule_id 0) ufw rules that nothing uses: anything
// but the current tunnel port, and the tunnel port itself for a protocol that
// has no direct forward left.
func (u *ufwManager) pruneShared() {
shared, err := u.db.ListUfwRulesByRule(0)
if err != nil || len(shared) == 0 {
return
}
forwards, err := u.db.ListForwardRules()
if err != nil {
return
}
used := map[string]bool{}
for _, f := range forwards {
if !f.NoUfw {
used[f.Proto] = true
}
}
port := u.tunnelPort()
for _, r := range shared {
if r.Port != port || !used[r.Proto] {
u.revokeRule(r)
}
}
}
// allowForward opens the host ports a forward rule needs: its host port (bound
// to the rule so it is revoked with it) plus the shared 8446 tunnel port for
// direct tcp/udp modes (bound to 0 — shared across rules).
// to the rule so it is revoked with it) plus the shared tunnel port for direct
// tcp/udp modes (bound to 0 - shared across rules). A rule made "without ufw"
// opens nothing, so nothing is revoked with it either.
func (u *ufwManager) allowForward(rule ForwardRule) {
switch rule.Proto {
case "udp":
u.allow(rule.HostPort, "udp", rule.ID)
if u.tunnelPort > 0 {
u.allow(u.tunnelPort, "udp", 0)
if !u.available() {
return
}
case "tcp":
u.allow(rule.HostPort, "tcp", rule.ID)
if u.tunnelPort > 0 {
u.allow(u.tunnelPort, "tcp", 0)
u.allowForwardPorts(rule)
}
default: // http long-poll listener is TCP
u.allow(rule.HostPort, "tcp", rule.ID)
func (u *ufwManager) allowForwardPorts(rule ForwardRule) {
if rule.NoUfw {
return
}
// "http" (long-poll) and "tcp" forwards both listen on TCP on the host.
hostProto, direct := "tcp", rule.Proto == "tcp"
if rule.Proto == "udp" {
hostProto, direct = "udp", true
}
// A listener bound to loopback is not reachable from outside anyway.
if ip := net.ParseIP(rule.ListenAddr); ip == nil || !ip.IsLoopback() {
u.allow(rule.HostPort, hostProto, rule.ID)
}
if port := u.tunnelPort(); direct && port > 0 {
u.allow(port, hostProto, 0)
}
}
// sync brings ufw in line with the saved forward rules: it opens what is
// missing (rules made while ufw or the integration was off, or a tunnel port
// that has since changed) and drops shared rules nothing uses.
func (u *ufwManager) sync() {
if !u.available() {
return
}
forwards, err := u.db.ListForwardRules()
if err != nil {
return
}
for _, f := range forwards {
u.allowForwardPorts(f)
}
u.pruneShared()
}
+89
View File
@@ -0,0 +1,89 @@
package server
import (
"encoding/json"
"net/http"
"strconv"
)
// GET /api/panel/users/{id}/agents - every machine, and which of them the
// user may use, for the editor of a user's machines.
func (s *Server) handleUserAgents(w http.ResponseWriter, r *http.Request) {
id, _ := strconv.ParseInt(r.PathValue("id"), 10, 64)
u, _ := s.db.AdminByID(id)
if u == nil {
http.NotFound(w, r)
return
}
agents, _ := s.db.ListAgents()
mine := s.db.UserAgents(id)
type item struct {
ID string `json:"id"`
Name string `json:"name"`
Hostname string `json:"hostname"`
Online bool `json:"online"`
On bool `json:"on"`
}
out := make([]item, 0, len(agents))
for _, a := range agents {
out = append(out, item{a.ID, a.Name, a.Hostname, s.online(&a), mine[a.ID]})
}
writeJSON(w, map[string]any{"username": u.Username, "agents": out})
}
// POST /api/panel/users/{id} - change a user: the role, the machines, the
// password sign-in, and whether they may switch it themselves. Fields left
// out stay as they are.
func (s *Server) handleUserEdit(w http.ResponseWriter, r *http.Request) {
me := s.currentAdmin(r)
id, _ := strconv.ParseInt(r.PathValue("id"), 10, 64)
u, _ := s.db.AdminByID(id)
if u == nil {
writeJSON(w, map[string]any{"ok": false, "error": "нет такого пользователя"})
return
}
var in struct {
Role *string `json:"role"`
Agents *[]string `json:"agents"`
PasswordOff *bool `json:"password_off"`
PwLocked *bool `json:"pw_locked"`
}
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 64<<10)).Decode(&in); err != nil {
writeJSON(w, map[string]any{"ok": false, "error": "плохой запрос"})
return
}
fail := func(msg string) { writeJSON(w, map[string]any{"ok": false, "error": msg}) }
if in.Role != nil {
role := *in.Role
if role != RoleAdmin && role != RoleUser {
fail("нет такой роли")
return
}
if role == RoleUser && u.IsAdmin() && s.db.CountRole(RoleAdmin) <= 1 {
fail("это последний администратор")
return
}
_ = s.db.SetUserRole(id, role)
s.db.Audit(me.ID, "", "user_role:"+u.Username+":"+role)
}
if in.Agents != nil {
_ = s.db.SetUserAgents(id, *in.Agents)
s.db.Audit(me.ID, "", "user_agents:"+u.Username+":"+strconv.Itoa(len(*in.Agents)))
}
if in.PasswordOff != nil {
if *in.PasswordOff && s.db.CountAdminKeys(id) == 0 {
fail("у пользователя нет ключей доступа - без пароля он не войдёт")
return
}
_ = s.db.SetPasswordOff(id, *in.PasswordOff)
action := "password_on"
if *in.PasswordOff {
action = "password_off"
}
s.db.Audit(me.ID, "", action+":"+u.Username)
}
if in.PwLocked != nil {
_ = s.db.SetPwLocked(id, *in.PwLocked)
}
writeJSON(w, map[string]any{"ok": true})
}
+15 -3
View File
@@ -7,6 +7,10 @@
set -eu
HOST="${HOST:-{{.Host}}}"
# The server's other address (the direct one, not behind a proxy like
# Cloudflare, or the main one when this script came from the direct one): the
# agent turns to it when HOST is out of reach.
FALLBACK_HOST="${FALLBACK_HOST:-{{.Fallback}}}"
{{if .Secret}}ENROLL_SECRET="${ENROLL_SECRET:-{{.Secret}}}"
{{end}}
BIN_PATH="/usr/local/bin/lpagent"
@@ -108,9 +112,9 @@ fi
# --- downloader -------------------------------------------------------------
download() { # download URL DEST
if command -v curl >/dev/null 2>&1; then
curl -fSL -o "$2" "$1"
curl -fSL --connect-timeout 15 -o "$2" "$1"
elif command -v wget >/dev/null 2>&1; then
wget -qO "$2" "$1"
wget -q -T 30 -O "$2" "$1"
else
red "Нужен curl или wget."; exit 1
fi
@@ -118,7 +122,14 @@ download() { # download URL DEST
bold "Скачивание агента ($ARCH) с $HOST…"
TMP="$(mktemp)"
download "$HOST/download/agent-linux-$ARCH" "$TMP"
if ! download "$HOST/download/agent-linux-$ARCH" "$TMP"; then
if [ -n "$FALLBACK_HOST" ] && [ "$FALLBACK_HOST" != "$HOST" ]; then
bold "$HOST недоступен, скачивание с $FALLBACK_HOST…"
download "$FALLBACK_HOST/download/agent-linux-$ARCH" "$TMP"
else
red "Не удалось скачать агента."; exit 1
fi
fi
install -m 0755 "$TMP" "$BIN_PATH"
rm -f "$TMP"
green "Бинарник установлен: $BIN_PATH"
@@ -129,6 +140,7 @@ chmod 700 "$CONF_DIR"
umask 077
cat > "$CONF_FILE" <<EOF
HOST=$HOST
FALLBACK_HOST=$FALLBACK_HOST
ENROLL_SECRET=${ENROLL_SECRET:-}
AGENT_NAME=$AGENT_NAME
STATE_PATH=$STATE_FILE
+239
View File
@@ -0,0 +1,239 @@
// The account button in the top bar: a dialog with who is signed in, their
// passkeys (add one, remove one), the password switch and signing out. The
// password can be turned off once there is a passkey to sign in with.
(function () {
'use strict';
var btn = document.getElementById('acct');
if (!btn) return;
var b64u = {
enc: function (buf) {
var s = '', b = new Uint8Array(buf);
for (var i = 0; i < b.length; i++) s += String.fromCharCode(b[i]);
return btoa(s).replace(/\+/g, '-').replace(/\//g, '_').replace(/=+$/, '');
},
dec: function (str) {
var s = atob(str.replace(/-/g, '+').replace(/_/g, '/') + '==='.slice((str.length + 3) % 4));
var b = new Uint8Array(s.length);
for (var i = 0; i < s.length; i++) b[i] = s.charCodeAt(i);
return b.buffer;
}
};
var esc = function (s) { return String(s).replace(/[&<>"]/g, function (c) { return {'&': '&amp;', '<': '&lt;', '>': '&gt;', '"': '&quot;'}[c]; }); };
var when = function (t) { return t ? new Date(t * 1000).toLocaleString() : 'ещё не использовался'; };
var csrf = (document.cookie.match(/(?:^|; )lpra_csrf=([^;]*)/) || [])[1] || '';
var X = '<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5" stroke-linecap="round"><path d="M18 6 6 18M6 6l12 12"/></svg>';
var KEY = '<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="7.5" cy="15.5" r="4.5"/><path d="m21 2-9.6 9.6M15.5 7.5l3 3L22 7l-3-3"/></svg>';
var dlg = document.createElement('dialog');
dlg.id = 'acctdlg';
dlg.tabIndex = -1;
dlg.innerHTML =
'<div class="dlg-h"><strong style="flex:1">Аккаунт</strong>' +
'<button type="button" class="btn ghost small icon close x" data-close title="Закрыть" aria-label="Закрыть">' + X + '</button></div>' +
'<div class="dlg-b acct">' +
' <div class="acct-who"><span class="acct-ava"></span><div class="acct-whotext"><div class="acct-name"></div><div class="muted acct-mode"></div></div>' +
' <form method="post" action="/logout" class="acct-out"><input type="hidden" name="csrf"><button type="submit" class="btn ghost">Выйти</button></form></div>' +
' <label class="acct-pw"><span><b>Вход по паролю</b><small class="muted acct-pwnote"></small></span>' +
' <input type="checkbox" class="sw" role="switch"></label>' +
' <label class="acct-pw acct-ai"><span><b>ИИ-ассистент</b><small class="muted">кнопка чата в правом нижнем углу</small></span>' +
' <input type="checkbox" class="sw" role="switch"></label>' +
' <details class="acct-chpw"><summary>Сменить пароль</summary>' +
' <form class="acct-pwform" autocomplete="on">' +
' <input type="text" name="username" autocomplete="username" hidden>' +
' <input type="password" name="current" placeholder="текущий пароль" autocomplete="current-password" required>' +
' <input type="password" name="new" placeholder="новый пароль, от 8 символов" autocomplete="new-password" minlength="8" required>' +
' <input type="password" name="again" placeholder="новый ещё раз" autocomplete="new-password" required>' +
' <button type="submit">Сменить</button></form></details>' +
' <details class="acct-pin"><summary>PIN-код <small class="acct-pinstate"></small></summary>' +
' <form class="acct-pinform">' +
' <input type="password" name="current" inputmode="numeric" maxlength="4" pattern="[0-9]{4}" placeholder="текущий PIN" autocomplete="off">' +
' <input type="password" name="pin" inputmode="numeric" maxlength="4" pattern="[0-9]{4}" placeholder="новый PIN, 4 цифры" autocomplete="off">' +
' <input type="password" name="again" inputmode="numeric" maxlength="4" pattern="[0-9]{4}" placeholder="новый ещё раз" autocomplete="off">' +
' <div class="row"><button type="button" class="btn danger acct-pindel">Убрать PIN</button><button type="submit" class="acct-pinsave">Сохранить</button></div>' +
' </form>' +
' <label class="acct-pingrace"><span>На этом устройстве не спрашивать после ввода</span><select>' +
' <option value="0">каждый раз</option><option value="5">5 минут</option><option value="15">15 минут</option>' +
' <option value="30">30 минут</option><option value="60">60 минут</option></select></label>' +
' <p class="acct-pinmsg" aria-live="polite"></p><p class="muted" style="font-size:12px;margin:8px 0 0">Спрашивается в каждой новой вкладке; при перезагрузке той же вкладки - нет.</p></details>' +
' <details class="acct-view"><summary>Вид</summary><div class="acct-viewin">' +
' <label><span>Значки вкладок</span><select data-pref="tabIcons"><option value="">слева</option><option value="right">справа</option><option value="hide">отключены</option></select></label>' +
' <label><span>Шрифт терминала</span><select data-pref="termFont" data-num><option value="0">обычный</option><option value="12">12</option><option value="13">13</option><option value="14">14</option><option value="15">15</option><option value="16">16</option><option value="18">18</option><option value="20">20</option></select></label>' +
' <label><span>Списки файлов</span><select data-pref="compact" data-bool><option value="">обычные</option><option value="1">компактные</option></select></label>' +
' <label><span>Время изменения файлов</span><select data-pref="fileTime"><option value="">показывать</option><option value="hide">скрыть</option></select></label>' +
' <label><span>Анимации</span><select data-pref="motion"><option value="">включены</option><option value="off">выключены</option></select></label>' +
' <label><span>Выбор модели в чате</span><select data-pref="aiModelPick"><option value="">когда моделей несколько</option><option value="always">всегда</option></select></label>' +
' <label><span>«Здесь» в меню «+»</span><select data-pref="openHere"><option value="">в папке активной вкладки</option><option value="off">в домашней папке</option></select></label>' +
' </div></details>' +
' <div class="acct-keysec">' +
' <h3>Ключи доступа</h3>' +
' <p class="muted acct-hint">Вход отпечатком, лицом, PIN-кодом устройства или аппаратным ключом. Ключей может быть несколько - например, телефон и ноутбук.</p>' +
' <ul class="acct-keys"></ul>' +
' <div class="acct-add"><input type="text" class="acct-keyname" maxlength="60" placeholder="название, например «телефон»">' +
' <button type="button" class="acct-addbtn">Добавить ключ</button></div>' +
' <p class="acct-msg"></p>' +
' </div>' +
'</div>';
document.body.appendChild(dlg);
dlg.querySelector('.acct-out [name=csrf]').value = decodeURIComponent(csrf);
var $ = function (sel) { return dlg.querySelector(sel); };
$('[data-close]').onclick = function () { dlg.close(); };
// Вид: each change applies at once and is kept on the server.
dlg.querySelectorAll('[data-pref]').forEach(function (sel) {
var k = sel.dataset.pref, cur = window.Prefs ? Prefs.get()[k] : '';
sel.value = sel.hasAttribute('data-bool') ? (cur ? '1' : '') : String(cur || (sel.hasAttribute('data-num') ? 0 : ''));
sel.onchange = function () {
var v = sel.hasAttribute('data-bool') ? sel.value === '1' : sel.hasAttribute('data-num') ? Number(sel.value) : sel.value;
if (window.Prefs) Prefs.set(k, v).then(function (ok) { if (!ok) msg('не сохранилось', true); });
};
});
var supported = !!(window.PublicKeyCredential && navigator.credentials && window.isSecureContext);
function msg(text, bad) {
$('.acct-msg').textContent = text || '';
$('.acct-msg').className = 'acct-msg' + (bad ? ' bad' : '');
}
function load() {
return fetch('/api/panel/account', {cache: 'no-store'}).then(function (r) { return r.json(); }).then(function (d) {
$('.acct-name').textContent = d.username;
dlg.querySelector('.acct-pwform [name=username]').value = d.username;
const pf = dlg.querySelector('.acct-pinform');
dlg.querySelector('.acct-pinstate').textContent = d.pin ? '· установлен' : '· не задан';
pf.current.hidden = !d.pin;
pf.querySelector('.acct-pindel').hidden = !d.pin;
pf.querySelector('.acct-pinsave').textContent = d.pin ? 'Сменить' : 'Установить';
$('.acct-pingrace').hidden = !d.pin;
try { $('.acct-pingrace select').value = localStorage.getItem('lpra.pin.grace') || '0'; } catch (e) { /* default */ }
$('.acct-ava').textContent = (d.username || '?').slice(0, 1).toUpperCase();
$('.acct-mode').textContent = d.password_off ? 'вход только по ключу доступа' : d.keys.length ? 'вход по паролю или ключу' : 'вход по паролю';
const sw = $('.acct-pw .sw');
sw.checked = !d.password_off;
sw.disabled = !d.keys.length || d.pw_locked;
$('.acct-pwnote').textContent = d.pw_locked ? 'закреплено администратором' + (d.password_off ? ': только по ключу' : '')
: !d.keys.length ? 'выключить можно, когда есть хотя бы один ключ'
: d.password_off ? 'выключен: войти можно только ключом' : 'можно выключить и входить только ключом';
var ul = $('.acct-keys');
ul.innerHTML = d.keys.length ? '' : '<li class="muted">Ключей пока нет.</li>';
d.keys.forEach(function (k) {
var li = document.createElement('li');
li.innerHTML = '<span class="acct-kicon">' + KEY + '</span><div class="acct-kinfo"><div>' + esc(k.name) + '</div>' +
'<div class="muted">добавлен ' + esc(when(k.created)) + ' · вход: ' + esc(when(k.last_used)) + '</div></div>' +
'<button type="button" class="btn danger small" title="Удалить ключ">Удалить</button>';
li.querySelector('button').onclick = async function () {
var last = d.keys.length === 1;
if (!await UI.confirm('Удалить ключ «' + k.name + '»?' + (last && d.password_off ? ' Это последний ключ: вход по паролю снова включится.' : ''), {ok: 'Удалить', danger: true})) return;
fetch('/api/panel/account/keys/' + k.id + '/delete', {method: 'POST'}).then(function (r) { return r.json(); })
.then(function (r) { if (!r.ok) msg(r.error, true); else { msg('ключ удалён'); load(); } });
};
ul.appendChild(li);
});
});
}
$('.acct-addbtn').onclick = function () {
if (!supported) { msg('этот браузер не умеет ключи доступа, или страница открыта не по https', true); return; }
var add = $('.acct-addbtn');
add.disabled = true;
msg('подтвердите на устройстве…');
fetch('/api/panel/account/keys/begin', {method: 'POST'}).then(function (r) { return r.json(); }).then(function (r) {
var o = r.options;
o.challenge = b64u.dec(o.challenge);
o.user.id = b64u.dec(o.user.id);
o.excludeCredentials.forEach(function (c) { c.id = b64u.dec(c.id); });
return navigator.credentials.create({publicKey: o});
}).then(function (cred) {
return fetch('/api/panel/account/keys/finish', {
method: 'POST', headers: {'Content-Type': 'application/json'},
body: JSON.stringify({
name: $('.acct-keyname').value,
clientDataJSON: b64u.enc(cred.response.clientDataJSON),
attestationObject: b64u.enc(cred.response.attestationObject)
})
}).then(function (r) { return r.json(); });
}).then(function (r) {
if (!r.ok) { msg(r.error, true); return; }
$('.acct-keyname').value = '';
msg('ключ добавлен - теперь вход по нему');
load();
}).catch(function (e) {
msg(e && e.name === 'NotAllowedError' ? 'отменено' : e && e.name === 'InvalidStateError' ? 'этот ключ уже добавлен' : 'не вышло: ' + (e && e.message || e), true);
}).then(function () { add.disabled = false; });
};
$('.acct-pwform').onsubmit = function (ev) {
ev.preventDefault();
const f = ev.target;
if (f.new.value !== f.again.value) { msg('новые пароли не совпадают', true); return; }
f.querySelector('button').disabled = true;
fetch('/api/panel/account/password/change', {method: 'POST', headers: {'Content-Type': 'application/json'},
body: JSON.stringify({current: f.current.value, new: f.new.value})})
.then(function (r) { return r.json(); })
.then(function (r) {
if (!r.ok) { msg(r.error, true); return; }
f.reset();
dlg.querySelector('.acct-chpw').open = false;
msg('пароль сменён');
})
.catch(function () { msg('ошибка сети', true); })
.then(function () { f.querySelector('button').disabled = false; });
};
function pinMsg(text, bad) {
$('.acct-pinmsg').textContent = text || '';
$('.acct-pinmsg').className = 'acct-pinmsg' + (bad ? ' bad' : '');
}
function savePin(pin) {
const f = $('.acct-pinform');
return fetch('/api/panel/account/pin', {method: 'POST', headers: {'Content-Type': 'application/json'},
body: JSON.stringify({pin: pin, current: f.current.value})})
.then(function (r) { return r.json(); })
.then(function (r) {
if (!r.ok) { pinMsg(r.error, true); return; }
f.reset();
// This tab stays unlocked: the PIN is for tabs opened from now on.
try { sessionStorage.setItem('lpra.pin.ok', '1'); } catch (e) { /* asked next load */ }
pinMsg(pin ? 'PIN сохранён' : 'PIN убран');
load();
})
.catch(function () { pinMsg('ошибка сети', true); });
}
$('.acct-pinform').onsubmit = function (ev) {
ev.preventDefault();
const f = ev.target;
if (!/^[0-9]{4}$/.test(f.pin.value)) { pinMsg('PIN - ровно 4 цифры', true); return; }
if (f.pin.value !== f.again.value) { pinMsg('новые PIN не совпадают', true); return; }
savePin(f.pin.value);
};
$('.acct-pingrace select').onchange = function (ev) {
const min = Number(ev.target.value);
try {
localStorage.setItem('lpra.pin.grace', String(min));
// Counted from now, as the PIN was entered to get here.
localStorage.setItem('lpra.pin.until', min ? String(Date.now() + min * 60000) : '0');
} catch (e) { pinMsg('браузер не даёт сохранить настройку', true); return; }
pinMsg(min ? 'на этом устройстве PIN не спросят ' + min + ' мин после ввода' : 'PIN спрашивается в каждой новой вкладке');
};
$('.acct-pindel').onclick = async function () {
if (await UI.confirm('Убрать PIN-код?', {ok: 'Убрать', danger: true})) savePin('');
};
// The assistant: each user's own switch; the chat button follows it.
fetch('/api/panel/ai', {cache: 'no-store'}).then(function (r) { return r.json(); }).then(function (d) { $('.acct-ai .sw').checked = !!d.on; }).catch(function () {});
$('.acct-ai .sw').onchange = function (ev) {
var on = ev.target.checked;
fetch('/api/panel/ai/toggle', {method: 'POST', headers: {'Content-Type': 'application/json'}, body: JSON.stringify({on: on})})
.then(function (r) { return r.json(); })
.then(function (r) { if (!r.ok) { ev.target.checked = !on; msg(r.error, true); } else document.dispatchEvent(new CustomEvent('lpra:ai', {detail: on})); });
};
$('.acct-pw .sw').onchange = async function (ev) {
const off = !ev.target.checked;
if (off && !await UI.confirm('Выключить вход по паролю? Войти можно будет только ключом доступа. Если ключи потеряются, их сможет сбросить другой администратор в «Настройках».', {ok: 'Выключить', danger: true})) {
ev.target.checked = true;
return;
}
fetch('/api/panel/account/password', {method: 'POST', headers: {'Content-Type': 'application/json'}, body: JSON.stringify({off: off})})
.then(function (r) { return r.json(); })
.then(function (r) { if (!r.ok) msg(r.error, true); else msg(off ? 'вход по паролю выключен' : 'вход по паролю включён'); load(); });
};
btn.onclick = function () {
msg('');
pinMsg('');
load().then(function () { dlg.showModal(); dlg.focus(); });
};
})();
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+132
View File
@@ -0,0 +1,132 @@
// The panel is one page: the header and three views (Машины, Терминал,
// Настройки). Moving between them changes the address without a reload, so
// the terminal view - and every session in it - lives as long as the tab.
// Машины and Настройки come from the server as fragments; their forms are
// sent from here and the fragment drawn again from the answer. A fragment's
// script is a module registered here, mounted on the fragment's root and
// unmounted - its timers, requests and listeners dropped - when it goes.
(function () {
'use strict';
const views = {'/': 'machines', '/workspace': 'terminal', '/terminal': 'terminal', '/settings': 'settings'};
const modules = {};
const mounted = new Map(); // area -> {mod, scope}
const $ = s => document.querySelector(s);
const viewOf = path => views[path] || null;
function register(name, mod) { modules[name] = mod; }
const scope = WSUtil.scope;
function unmount(area) {
const m = mounted.get(area);
if (!m) return;
mounted.delete(area);
try { if (m.mod.unmount) m.mod.unmount(m.root); } finally { m.scope.end(); }
}
function mount(area) {
const root = area.querySelector('[data-module]');
const mod = root && modules[root.dataset.module];
if (!mod) return;
const sc = scope();
mounted.set(area, {mod, scope: sc, root});
mod.mount(root, sc);
}
// put draws html in area, the old module unmounted and the new one mounted.
function put(area, html) {
unmount(area);
area.innerHTML = html;
mount(area);
}
const failHTML = msg => '<div class="wrap"><p class="err">' + msg + ' <button type="button" class="btn ghost small" data-retry>Повторить</button></p></div>';
function withFrag(url) { const u = new URL(url, location.href); u.searchParams.set('frag', '1'); return u; }
// A request that ended on the sign-in page: the session is over.
function signedOut(r) { return r.redirected && new URL(r.url).pathname === '/login'; }
async function load(area, url) {
let r;
try { r = await fetch(withFrag(url), {credentials: 'same-origin', cache: 'no-store'}); } catch (e) { r = null; }
if (r && signedOut(r)) { location.href = '/login'; return; }
if (!r || !r.ok) {
put(area, failHTML(r ? 'не загрузилось: ошибка ' + r.status : 'нет связи с сервером'));
area.querySelector('[data-retry]').onclick = () => load(area, url);
return;
}
put(area, await r.text());
}
function show(view) {
document.querySelectorAll('main#app > .view').forEach(v => { v.hidden = v.dataset.view !== view; });
document.querySelectorAll('[data-nav]').forEach(a => a.classList.toggle('on', a.dataset.nav === view));
$('main#app').dataset.view = view;
document.dispatchEvent(new CustomEvent('app:view', {detail: view}));
}
// The address the shell shows; a history step that keeps it (a chat on a
// phone closing by «назад») moves no view.
let shown = location.pathname + location.search;
async function go(url, push = true) {
const u = new URL(url, location.href);
const view = viewOf(u.pathname);
if (!view || u.origin !== location.origin) { location.href = u.href; return; }
if (push && u.pathname + u.search !== location.pathname + location.search) history.pushState({}, '', u.pathname + u.search + u.hash);
shown = u.pathname + u.search;
show(view);
if (view !== 'terminal') await load($('.view[data-view="' + view + '"]'), u.pathname + u.search);
}
// Links to a view move to it; any other link behaves as a link.
document.addEventListener('click', ev => {
const a = ev.target.closest('a[href]');
if (!a || ev.defaultPrevented || ev.button !== 0 || ev.metaKey || ev.ctrlKey || ev.shiftKey || a.target || a.hasAttribute('download')) return;
const u = new URL(a.href, location.href);
if (u.origin === location.origin && viewOf(u.pathname)) { ev.preventDefault(); go(u.href); }
});
window.addEventListener('popstate', () => { if (location.pathname + location.search !== shown) go(location.href, false); });
// A form in a fragment is sent from here: the answer's address says what
// to draw. /logout and forms outside the views go the usual way; a form
// whose onsubmit said no (a confirm) is not sent.
document.addEventListener('submit', async ev => {
const f = ev.target;
const area = f.closest('.view, .frag-area');
if (!area || ev.defaultPrevented || (f.method || '').toLowerCase() !== 'post' || /\/logout$/.test(f.action)) return;
ev.preventDefault();
let r;
try { r = await fetch(f.action, {method: 'POST', body: new FormData(f), credentials: 'same-origin'}); }
catch (e) { alert('Нет связи с сервером'); return; }
if (signedOut(r)) { location.href = '/login'; return; }
const to = new URL(r.url);
// From a dialog to a view (a machine deleted): the dialog closes.
if (!area.classList.contains('view') && viewOf(to.pathname)) {
const d = area.closest('dialog');
if (d) d.close();
go(to.href);
return;
}
if (area.classList.contains('view')) {
const v = viewOf(to.pathname);
if (v && v !== area.dataset.view) { go(to.href); return; }
history.replaceState({}, '', to.pathname + to.search + to.hash);
}
// The answer is the page whole (the shell): ask for its fragment.
await load(area, to.pathname + to.search);
if (to.hash) { const t = area.querySelector(to.hash); if (t) t.scrollIntoView(); }
});
// Leaving the page drops the terminals: the browser asks while one is alive.
window.addEventListener('beforeunload', ev => {
if (window.Workspace && window.Workspace.hasLiveSessions()) { ev.preventDefault(); ev.returnValue = ''; }
});
// start mounts what the server drew, once every script has registered.
function start() {
// The old address of the workspace shows as the new one.
if (location.pathname === '/terminal') { history.replaceState({}, '', '/workspace' + location.search); shown = '/workspace' + location.search; }
document.querySelectorAll('main#app > .view').forEach(mount);
show({dashboard: 'machines', settings: 'settings'}[$('main#app').dataset.view] || 'terminal');
// /agents/{id}/terminal or /files: that pane, at once.
const o = window.APP && window.APP.open;
if (o && window.Workspace) window.Workspace.open(o.kind, o.agent);
}
window.App = {go, load, register, put, start, scope};
})();
@@ -0,0 +1,21 @@
MIT License
Copyright (C) 2017 by Marijn Haverbeke <marijn@haverbeke.berlin> and others
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
Loaded 100 of 134 files, more files were not shown because too many files have changed in this diff. Show more