sqoob

A self-hosted booru with a Rust core. So far it has:

• posts, tags, tag aliases, implications, and categories, and Shimmie2's search, with its metatags, sort orders, and tag lists, on SQLite, PostgreSQL, and MariaDB;
• uploads of JPEG, PNG, GIF, and WebP images, validated and thumbnailed by a separate worker process;
• accounts with roles and permissions, browser sessions, and API tokens;
• ratings, with the ratings each role may see and each account's own hiding and blurring;
• moderation: post sources, locked posts, a trash for deleted posts, and purging;
• comments, and a wiki that keeps every revision, both written in Markdown;
• favorites, votes and scores with popular pages, parent and child posts, pools, notes on images, and reports to moderators, as in Shimmie2;
• an edit history of every post's tags, source, rating, and parent, with reverts of one post or of everything an account changed since a time;
• Shimmie2's tag box terms, such as parent:12, pool:Sketches, and vote:up, when editing a post and when uploading one;
• a public profile for every account, with a biography and an avatar, and a list of accounts;
• site settings and features that administrators change while the site runs, as with Shimmie2's board config;
• replacing posts' files, converting them to other formats, and regenerating thumbnails;
• bulk edits of a search's posts, and Shimmie2's Bulk Add of a folder of files on the server;
• view counts, as Shimmie2's view counter keeps them;
• a home page, blocks, head additions, custom CSS, and a site icon, none of which can run a script, and robots.txt and a sitemap, as in Shimmie2;
• email for confirming email addresses and resetting passwords;
• a web interface for browsing, searching, tagging, uploading, account settings, and administration, which works without JavaScript;
• a GraphQL API.

docs/ontology.md defines the concepts sqoob is built from and what each one is called. Every permission check lives in the operations, the functions that take the viewer, rather than in the GraphQL API or the command line, so every interface enforces the same rules.

Building needs Rust 1.94 or newer. Build inside the repository: its .cargo/config.toml builds the bundled SQLite to read double-quoted names only as names, and sqoob refuses to use SQLite built otherwise.

The browser script is TypeScript, built with esbuild, which needs Node.js 22 or newer. Run npm ci once, then npm run build before building sqoob, which embeds what it writes to assets/dist. Without it, sqoob still builds, with a warning, and its pages run no script; they work without one.

Running

cargo run

Variable	Default	Meaning
SQOOB_DATABASE_URL	sqlite://sqoob.db	A sqlite:, postgres:, or mariadb: URL.
SQOOB_LISTEN	127.0.0.1:8080	IP address and port to serve on.
SQOOB_MEDIA_DIR	media	Where uploaded files and thumbnails are stored.
SQOOB_WORK_DIR	work	Scratch space for uploads in progress. Keep it outside the media directory.
SQOOB_TRUSTED_PROXIES	none	Reverse proxies whose X-Forwarded-For header names the client, such as 127.0.0.1, ::1 or 10.0.0.0/8.
SQOOB_MAIL_URL	none	How mail is sent; see Mail. Without it, the site sends no mail.
SQOOB_MAIL_FROM	none	The email address mail comes from, such as Booru <booru@example.com>. Needed with SQOOB_MAIL_URL.
SQOOB_PUBLIC_URL	none	The site's public URL, such as https://booru.example.com, for links in mail and the sitemap. Pages call the site by its host, or sqoob without it, unless the site name setting gives another name.
SQOOB_BULK_ADD_DIR	none	The folder whose files Bulk Add may add as posts; see Bulk edits and Bulk Add. Without it, Bulk Add is unavailable.

These take effect when sqoob starts, so changing one needs a restart. The site settings that administrators change in the browser take effect at once. A new version of sqoob migrates the database itself when it first starts.

The first start claims the media directory for this installation by writing a .sqoob-installation marker. sqoob refuses to use a media directory that belongs to another installation, sits inside one, or already holds files without a marker.

Serve the site over HTTPS. Session cookies are marked Secure, and browsers only make an exception for localhost. Behind a reverse proxy, list the proxy in SQOOB_TRUSTED_PROXIES, so that limits on failed sign-ins count each client's own IP address rather than the proxy's.

Accounts

The first administrator comes from the command line, which needs only the configuration above:

sqoob account create jey --role Administrator

This prints a one-time code. Within 24 hours, enter it at /reset-password, or send it with the resetPassword mutation, to set the account's password, then sign in at /sign-in. The same commands recover access when no one can sign in:

Command	Purpose
sqoob account create NAME [--role ROLE]	Creates an account and prints a code that sets its password.
sqoob account reset NAME	Prints a code that sets an account's password.
sqoob account role NAME ROLE	Gives an account another role.
sqoob account enable NAME	Lets a disabled account sign in again.
sqoob mail test EMAIL	Sends a test email with the mail configuration.
sqoob bulk-add FOLDER [TAG...]	Adds the files in a folder of the bulk add folder as posts, as Shimmie2's bulk-add command does; . is all of it. The server adds them in the background.

Every account has a role, and roles grant permissions: UPLOAD, EDIT_METADATA (tags, sources, and parents), EDIT_RATINGS, FAVORITE, VOTE, EDIT_NOTES (adding, changing, deleting, and asking for notes), EDIT_POOLS (creating pools, and changing one's own and open ones), REPORT, SEE_VIEW_COUNTS, MODERATE (locking posts and their comments, changing their uploaders, reading and dismissing reports, removing votes, and reverting everything an account changed since a time), DELETE_POSTS (and restoring them), PURGE_POSTS, EDIT_FILES (replacing and converting posts' files, and regenerating thumbnails), BULK_EDIT (starting bulk edits, each of whose changes still needs its own permission), BULK_ADD, COMMENT, DELETE_COMMENTS (anyone's), EDIT_WIKI, MANAGE_WIKI (locking and deleting pages and revisions), MANAGE_POOLS (changing and deleting anyone's pools), MANAGE_TAGS (aliases, implications, renames, lowercasing, and categories), CUSTOMIZE (the home page, blocks, head additions, custom CSS, and the site icon), INVITE, MANAGE_ACCOUNTS, MANAGE_ROLES, and ADMINISTER, which allows everything. There are three built-in roles, which can be renamed but not deleted:

• Everyone holds what every viewer may do and see, visitors included. It starts able to do nothing, and it can be allowed to upload, edit metadata, edit notes, report posts, see view counts, comment, and edit the wiki, never more. Accounts get it on top of their own role, so signing in never takes anything away.
• Member is the role new accounts get. It starts with uploading, editing metadata and ratings, favoriting, voting, editing notes and pools, reporting posts, commenting, and editing the wiki. Migration 14 gave the Member role of an existing site the permissions that came with it: favoriting, voting, editing notes and pools, and reporting.
• Administrator always has ADMINISTER and sees every rating.

Administrators can define further roles. An account that manages accounts or roles acts only on accounts and roles whose permissions it holds and whose ratings it may see itself, so it cannot grant more than it has.

Registration is closed by default: only account managers create accounts, at /admin/accounts or with createAccount, which returns a code for the new account's first password. An administrator can open registration to anyone or to holders of invitations, at /admin/site or with setRegistrationMode; visitors then register at /register.

An account can have an email address, which it sets with setEmail and its password. The address gets a code, and it becomes the account's address once the code comes back through confirmEmail; until then it is never used. Anyone who has lost their password can ask for a reset with requestPasswordReset, giving their account name or email address. The reset code goes only to a confirmed address, works for an hour, and goes out at most once every 15 minutes. The request gets the same answer whether or not an account matches, so it reveals nothing about who has an account. Managers can also issue reset codes with issuePasswordReset, which works without email.

A browser signs in at /sign-in or with signIn, either of which sets a session cookie. Sessions end after 30 days unused or a year in all, when they are revoked, and when the password changes. Failed sign-ins are limited per account and per IP address. Scripts use personal API tokens instead, made with createApiToken and sent as Authorization: Bearer <token>. A token acts with the permissions in its scopes that its account holds at the time, and with whatever visitors may. It is shown once, and it cannot manage the account it belongs to.

Every change made through the permission to manage accounts, roles, tags, pools, or the wiki, to moderate, delete, or purge posts, to replace files, to delete comments, to customize, or to administer the site, leaves an audit record: who made it, from which IP address, when, and what it changed, naming accounts, roles, and tags as they were named then. Changes made from the command line are recorded too, with the operator as the actor. An account changing its own sessions, tokens, or invitations leaves no record, and neither does a change that fails or changes nothing. Records never hold a code or a token. Administrators read them at /admin/audit, or with auditRecords, narrowed to what one account did or to what was done to an account, a role, or a post; nothing changes or deletes them.

Ratings

Every post has one of Shimmie2's ratings: SAFE, QUESTIONABLE, EXPLICIT, or UNRATED. Posts are unrated unless createPost, finishUpload, or setPostRating gives them a rating, which needs EDIT_RATINGS. A rating decides who may see a post, so members have that permission and visitors can never be given it. Each role has the ratings its accounts may see, and every role sees every rating until an administrator narrows it with updateRole. Accounts also see whatever the Everyone role may, and accounts whose role has ADMINISTER see everything.

A post with a rating the viewer may not see is restricted for that viewer: it is left out of searches and cannot be edited. Asking for it directly fails with UNAUTHENTICATED for visitors and FORBIDDEN for accounts, its page answers 403 Forbidden and invites visitors to sign in, and its files answer 404 Not Found.

Each account can also hide ratings from its own listings and ask for ratings to be blurred, with setRatingPreferences. Hiding only sets what listings show by default: a search that names a rating lists it anyway, and a link to a post with a hidden rating still opens the post. A search that only excludes ratings, such as -rating:q, narrows the usual listing and still leaves hidden ratings out. A listing counts what it left out in two numbers: restrictedCount for matching posts the site keeps from the viewer, and hiddenCount for those the viewer's own hiding left out. The pages blur a thumbnail until the pointer rests on it or it has the keyboard's focus, and a post's image until the viewer chooses to show it; other clients do their own blurring.

Searches choose ratings with rating: and a rating's name, or Shimmie2's letters for one or more ratings, where ? stands for unrated: rating:safe, rating:sq for safe or questionable posts, rating:?, rating:* for every rating the viewer may see, hidden ones included, and -rating:e for everything but explicit posts.

Files that visitors may see are sent with Cache-Control: public, max-age=3600, so a shared cache in front of the site, such as a CDN, may keep serving one for up to an hour after its post is restricted. Other files are sent private, so only the viewer's own browser keeps them.

An administrator can turn ratings off, as Shimmie2 can, at /admin/site or with setFeature. While they are off, every post counts as safe: whoever may see safe posts sees every post, and whoever may not sees none. Nothing shows, asks for, searches by, or changes a rating, and accounts' hiding and blurring don't apply. Posts keep their ratings, which count again when ratings are turned back on, as do roles' ratings and accounts' preferences. A role manager still grants only the ratings they could see with ratings on. Turning ratings back on restricts posts again, so the caching above applies to them.

Moderation

A post's source says where its image came from: a web address, which its page links, or a few words. Like tags, it is set with EDIT_METADATA, on upload, at the post's page, or with setPostSource.

Moderators, with MODERATE, lock posts with lockPost, after which only moderators can change a post's tags, source, rating, parent, or notes; tag rules still reach it. They also credit posts to other accounts, or to none, with setPostUploader.

deletePost, with DELETE_POSTS, moves a post to the trash. A deleted post is in no listing, and to anyone without DELETE_POSTS or PURGE_POSTS it is as if it never existed: its page, its files, and its download answer 404 Not Found. It keeps its tags, which go on counting it, and its files, and nothing about it changes until restorePost brings it back as it was. Those who may delete or purge posts list the trash with the search in:trash, Shimmie2's term, or through "Deleted posts" in administration. Uploading a deleted post's file again is refused as a duplicate.

purgePost, with PURGE_POSTS, removes a deleted post for good: its tags, so their counts drop, its record, and then, in a background job, the stored files that no other post has. Its ID is never used again, and its page asks for confirmation first. Locking, changing uploaders, deleting, restoring, and purging all leave audit records.

Files

Those with EDIT_FILES change posts' files, each change as a background job:

• Replacing, at /posts/{id}/replace or with replacePostFile, gives a post a new file and keeps everything else about it, as Shimmie2's replace file does. A file that is already another post's, or the post's own, is refused. Each replacement leaves an audit record.
• Converting, from a post's page or with convertPost, changes the file it shows and offers to JPEG, PNG, or lossless WebP, as Shimmie2's transcode does. A conversion is always made from the original, which stays, so discardConversion goes back to it without loss, and an animation is never made a still. The site settings give the quality of JPEG conversions and the color that transparency becomes. Only those with EDIT_FILES see a converted post's original.
• Regenerating thumbnails makes them again from the files: one post's from its page, a search's posts in a bulk edit, and every post's, or only the missing ones, at /admin/files, 25 posts to a job.

A file that a change replaces stays for a day, for pages that still show it, and is then removed.

Bulk edits and Bulk Add

/posts/edit, linked from every listing for those with BULK_EDIT, changes every post of the search, or the posts chosen from it, as Shimmie2's bulk actions do: adding and removing tags, or giving exactly these, setting the source, the rating, or the parent, chaining the posts as each other's parents, deleting, restoring, locking, unlocking, adding them to a pool or to a new one, favoriting, regenerating thumbnails, and converting. bulkEdit does the same. Each change needs its own permission too. The work runs as background jobs of 100 posts, or 10 for work on files, acting with the permissions and visibility its starter had when it started; a post a change cannot be made to is listed in the results, and the rest change. Those who manage tags also lowercase every tag at once, at /admin/tags or with lowercaseTags.

Bulk Add, with BULK_ADD, adds every file in a folder of the bulk add folder that SQOOB_BULK_ADD_DIR names, and in its subfolders, as posts by whoever starts it, as Shimmie2's Bulk Add does. /admin/bulk-add and bulkAddPreview list what a folder would add, in the order Shimmie2 adds it, with the tags each file would get: those of the folders it is in, each folder's name being tags separated by spaces, with ; for : and __ for a space, and a folder whose name ends in a colon, such as character;, giving its category to the tags of the folder inside it; those of a file named like 123 - tags.png; and those given for every file, which may hold tag box terms. Starting it there, with startBulkAdd, or with sqoob bulk-add queues a job for each file, behind all other work, and its report says what became of each file: a new post, a post that already had it, which is how adding a folder again adds only what is new, or why it failed. Hidden files and folders, whose names start with a dot, and links are passed over, and the files stay where they are.

View counts

A post's page counts a view once per IP address an hour, as Shimmie2's view counter does. Only the totals are kept, never who viewed what. As in Shimmie2, only those with SEE_VIEW_COUNTS see the counts, on posts' pages and as viewCount, which means administrators unless a role is given it, but anyone may list posts by them with order:views, as through the listing's "Most viewed" link. View counts are a feature, which administrators may turn off.

Customization

Those with CUSTOMIZE change what the site adds to its pages, at /admin/customize and /admin/blocks, or with setCustomization, setSiteIcon, removeSiteIcon, createBlock, updateBlock, and deleteBlock:

• The home page, /home, as Shimmie2's: the site's name, links to its main pages or others, a search box, text, the post count in large digits, and the contact link. The front page setting makes it what / shows.
• Head additions: <meta> and <link> elements for every page's head, such as fonts, stylesheets, and verification tags, as Shimmie2's custom HTML headers, without scripts.
• Custom CSS, which pages load after the site's own stylesheet, from /site.css.
• The site icon, a PNG, GIF, JPEG, or WebP image of at most 512 by 512 pixels, served as /favicon.ico.
• Blocks, as Shimmie2's: HTML with a title, beside the page's content, above it, or below it, on the pages whose paths match their patterns, such as /posts*, for everyone, for visitors, or for accounts with chosen roles, in order of their positions.

HTML is sanitized when it is saved, and saving says what it took out: scripts, styles, event handlers, forms, and links that are not web or email addresses. An <iframe> stays only from an allowed embed origin, sandboxed so that it cannot navigate the page. None of it shows on administration or on the pages that show credentials, such as signing in and account settings. The pages that show it allow, in their Content Security Policy, the image, stylesheet and font, and embed origins that administrators allow in the site settings, and still run no script but the site's own. Each change leaves an audit record.

Crawlers

/robots.txt has Shimmie2's rules: a three-second crawl delay, and the edit history of tags, sources, and notes kept out of search engines. Asked for under another host than SQOOB_PUBLIC_URL's, it keeps crawlers out altogether, as Shimmie2's canonical domain does, so that the site is indexed under one name. With SQOOB_PUBLIC_URL set and the sitemap setting on, /sitemap.xml lists the front page, the tags by how many posts have them, and the posts visitors may see, newest first, as Shimmie2's sitemap does, and robots.txt says where it is. The sitemap is made at most once an hour.

Tag categories

As in Shimmie2, a tag's name decides its category: character:saturn is in the category whose prefix is character. Those who manage tags make categories at /admin/tags/categories or with createTagCategory, each with a singular and a plural name, a color, and a position. A prefix cannot change, and cannot be the name of a search term such as rating or order. Deleting a category leaves its tags as they are, uncategorized.

A post's page groups its tags by category, in the categories' order, under each category's plural name. The tags beside a post or a listing show a categorized tag in its category's color and without its prefix, as Shimmie2 does; tag fields, the tag cloud, suggestions, and the tag lists keep whole names, in color, except under a category's own heading. The colors come from /tag-categories.css, made from the categories, since pages allow no inline styles. In the API, tags carry their category, and tagCategories lists them all.

Search

Searches use Shimmie2's syntax, and each means the same on SQLite, PostgreSQL, and MariaDB. Tags separated by spaces must all match; -tag leaves a tag out, ~a ~b asks for at least one of them, and * matches any characters. Searching for an alias finds the tag it stands for.

Metatags filter by what a post is rather than its tags:

Metatag	Matches
id=1234, id=1,2,3, id>1234	Post IDs.
posted=2020-01-31, posted>=2020-01, posted<2020	When the post was posted: a day, a month, or a year, in UTC.
user=name, poster=name, user_id=12	Who uploaded it.
source=pixiv.net, source=any, source=none	Text the source contains, without http: or https: and whatever the case of its letters A to Z, or whether it has one.
width>=1920, height<720, size>=1920x1080, ratio=16:9	The image's dimensions and shape.
filesize>2m	The file's size, in bytes or with k, m, or g for KiB, MiB, or GiB.
ext=png, mime=image/png, md5=…	The file's format and MD5 hash.
tags>=10, charactertags>=2	How many tags the post has, or how many in the category with that prefix.
comments>=3, commented_by=name, commented_by_userno=12	How many comments the post has, and who commented on it.
favorites>=5, favorited_by=name, favorited_by_userno=12	How many accounts favorited the post, and who did.
score>=3, score<-1, upvoted_by=name, downvoted_by=name, upvoted_by_id=12	The sum of the post's votes, and who voted it up or down.
parent=12, parent=any, parent=none, child=any, child=none	The post's parent, and whether it has children.
pool=12, pool=any, pool=none, pool_by_name=sketch_book	The pools the post is in.
note=text, notes>=2, notes_by=name, notes_by_userno=12	Text in the post's notes, how many it has, and who wrote them.
rating:sq, in:trash	Ratings, as above, and the trash, as below.

Comparisons are = or :, and <, <=, >, and >= for numbers and dates. A - before a filter, or != in it, keeps the posts it does not match. A term that reads as a metatag is never a tag, and Shimmie2's metatags that sqoob does not have yet, such as filename and length, are refused rather than taken for tags.

Searches list the newest posts first. order:id, order:posted, order:width, order:height, order:filesize, order:favorites, order:score, and order:views sort otherwise, oldest or largest first unless _asc or _desc follows, and posts that sort the same go by ID. order:random_1234 shuffles the posts the same way on every page; the listing gives order:random a seed of its own, and order:dailyshuffle shuffles differently each day. /help/search explains every term.

The tag lists are at Shimmie2's addresses: /tags/map, which sizes the most used tags by their use, /tags/alphabetic, /tags/popularity, and /tags/categories, with each category's most used tags. Each takes Shimmie2's starts_with and mincount, and the API lists tags the same way with tags.

Comments and wiki

Wiki pages and comments are written in Markdown, with what boorus add: [[page]] and [[page|text]] link wiki pages, >>123 links post 123, ||text|| hides a spoiler, and every line break shows, as on Shimmie2. A > starts a quote only with a space after it. Raw HTML shows as the text it is, links go only to web and email addresses and to the site, and images show only from the site; an image from elsewhere becomes a link to it. /help/formatting shows it all. Imported Shimmie2 BBCode will be converted to Markdown.

Comments go under each post, oldest first. /comments lists the posts with the newest ones, and every listing shows the five newest beside its posts, as Shimmie2 does; the site settings change both numbers. Writing one needs COMMENT, which members have and visitors may be allowed. Authors change and delete their own comments, and DELETE_COMMENTS deletes anyone's. Moderators lock a post's comments, after which only moderators comment on it. As in Shimmie2, a post takes the same comment only once, a comment that says one thing over and over is refused, and an IP address may send 30 comments in 15 minutes. A deleted post's comments are hidden with it, and purging it removes them.

A wiki page's title works like a tag's name, so Black Cat and black_cat are one page, and the page named after a tag describes it: it shows the tag beside its text, tags link it with a "?", and a search for that tag alone starts with its first paragraph. Renaming or merging a tag takes its page along, unless the tag it becomes has one. Every save keeps a revision, and names the revision it was edited from, so a save that would overwrite someone else's change is refused and shows the page as it is now. Pages are at Shimmie2's addresses: /wiki/<title>, with ?revision=, /wiki/<title>/history, and /wiki/<title>/diff?r1=…&r2=…. EDIT_WIKI creates and edits pages; MANAGE_WIKI locks them against other editors, deletes revisions such as vandalism, and deletes pages. /wiki lists the pages, below the page titled Index if there is one, and Shimmie2's /wiki/wiki:list leads there. Two pages show on others, as in Shimmie2: wiki:sidebar beside every page and the list, and wiki:default in place of a page that does not exist. They are edited like any page, so lock them if only managers should change them.

Favorites and votes

Accounts favorite posts with FAVORITE and vote them up or down with VOTE, on a post's page or with favoritePost, unfavoritePost, and setPostVote; members may do both, and visitors neither. A post's page shows how many accounts favorited it and which, as Shimmie2 does, and its score, the sum of its votes. A profile counts the account's favorites and votes, linking each to a search. Moderators see every vote on a post at /posts/{id}/votes and remove a post's votes there, or all of an account's from its profile, which leaves an audit record. /popular/day, /popular/month, and /popular/year list the posts uploaded on a day, in a month, or in a year with the highest scores other than 0, as Shimmie2's popular pages do, and lead to the periods before and after.

Parent and child posts

A post can have a parent, such as the original of an edited version, as in Shimmie2. The parent is metadata: it is set with EDIT_METADATA, in the edit form's Parent field, with a tag box term, or with setPostParent, and it must be a post the editor may see that is not deleted, never the post itself or one that descends from it. A post's page shows its parent, siblings, and children that the viewer may see, and searches find them with parent= and child=. Purging a parent leaves its children without one.

Edit history

Every save of a post's tags, source, rating, or parent is an edit: how the save left them, who made it, and when. Uploading is a post's first edit, and migration 14 recorded every existing post as it was then, without an editor. A save that changes nothing records nothing, and edits follow tags through renames and merges. Whoever may see a post reads its history at /posts/{id}/history, or with the post's edits; /history and edits list recent edits, of everyone or of one account, for the posts the viewer's listings show. No IP addresses are kept.

Reverting sets a post back to how an edit left it, which is a new edit, with revertPost or the history's buttons; it needs the permissions the changes need, and leaves alone what belongs to a feature that is off. Moderators revert everything one account, or visitors, changed since a day, from the list of that account's edits, which its profile links, or of visitors' edits, which any edit by a visitor links, or with revertEdits, as Shimmie2's mass revert does: each post goes back to its latest edit by someone else from before then. The page and postsEditedSince say how many posts that touches first; the revert then runs as background jobs of 100 posts, and leaves an audit record. History goes on recording while the feature is off, so turning it back on shows nothing missing.

Pools

A pool is a titled, ordered collection of posts, such as the pages of a comic, with a description in Markdown, as in Shimmie2. /pools lists them, and anyone may see a pool at /pools/{id}, with the posts in it that their listings would show; a post's page shows its pools with the posts before and after it in each. EDIT_POOLS creates pools. A pool's creator, and those with MANAGE_POOLS, rename it, open or close it, and delete it; an open pool, which Shimmie2 calls public, lets anyone who may edit pools change its description and its posts, and a closed one changes only through its creator and pool managers. Posts are added by ID, from a search, from a post's page, or with a tag box term, and go at the end; the pool's order page numbers them, and they can be taken out or turned around. Every change to a pool's posts is recorded at /pools/changes, where it can be reverted, and a pool manager's change to someone else's pool leaves an audit record. In the API: pools, pool, poolChanges, createPool, updatePool, deletePool, addPoolPosts, removePoolPosts, orderPool, reversePool, and revertPoolChange.

Notes

Notes are text on rectangles of a post's image, such as translations, written in Markdown, as Shimmie2's notes. The rectangles show over the image and link to the notes listed below it, and with the browser script each shows its note when the pointer rests on it. EDIT_NOTES adds, moves, changes, and deletes notes, on a locked post only with MODERATE too: in forms under the image, where the script lets the rectangle be drawn on the image, or with createNote, updateNote, and deleteNote. Every change is a numbered revision of its note, deleting included, so /notes/{id}/history and revertNote bring back any earlier state. Accounts ask for notes on a post, and those who edit notes clear the requests. /notes lists the posts with notes, /notes/requests those asked for, and /notes/changes the newest changes.

Reports

REPORT, which members have and visitors may be allowed, reports a post to moderators with a reason of up to 300 characters, as Shimmie2's post reports do. An account reports a post once, and an IP address may send 10 reports in 15 minutes. Moderators read the reports at /admin/reports and on posts' pages, and dismiss one, a post's, or an account's, which leaves an audit record. A deleted post's reports leave the list with it, and come back if it is restored. In the API: reports, reportPost, dismissReport, dismissPostReports, and dismissAccountReports.

Tag box terms

As in Shimmie2's tag box, some entries among a post's tags change the post instead of becoming tags:

Term	Does
parent:12, parent:none	Sets the post's parent, or leaves it without one.
child:12	Makes post 12 a child of this one; that needs what editing post 12 needs.
pool:12, pool:Sketch_Book, pool:lastcreated	Adds the post to the end of a pool, by its ID or title, or to your newest pool. pool:12:3 puts it third. A post already in the pool stays where it is.
vote:up, vote:down, vote:remove	Votes on the post, or takes your vote back.
rating:s, rating:safe	Sets the rating, by letter or by name.
source:https://…, source:none	Sets the source, or clears it.

= works as well as :, and the names' case does not matter. A term counts instead of the edit form's field for the same thing, each needs what its change needs, and an edit makes all its changes or none. Uploads take the same terms, checked when the upload is finished and done when its post is published, so pool:lastcreated means the pool that was newest when the upload finished. A term that cannot be read, such as vote:sideways, is refused and says why. Since these names make terms, nothing named like one can be a tag.

Profiles

Every account has a profile at /accounts/<name>, as Shimmie2 has a user page: its role, when it joined, and how many posts it uploaded and comments it wrote, with how many a day, its newest comments, and what its owner wrote about it. Anyone may read it. Its counts and comments cover what the viewer's listings would show, so they never reveal a post the viewer may not see. /accounts/<name>/comments lists all its comments beside their posts, and /accounts lists every account by name, narrowed to names that start a certain way or to a role, as Shimmie2's user list does.

An account writes its biography in Markdown on its profile, and chooses its avatar on a post's page: the post's thumbnail, cropped square, shown on its profile and beside its comments. An avatar shows only to viewers whose listings would show its post, blurred for those who blur its rating, and purging the post removes it. Owners change their profiles through a signed-in session, not an API token. Those who manage an account change its biography and remove its avatar too, which leaves an audit record. Names link to profiles wherever they show: beside comments, as a post's uploader, and in the wiki's history.

Site settings

Administrators change how the site presents itself and which features it has while it runs, on one form at /admin/site, as with Shimmie2's board config, or with the API. A change shows from the next page on. Each change leaves an audit record; one that sets what is already set changes nothing, and one value that a setting cannot take keeps the whole form from saving.

Setting	Default	Meaning
Registration	Closed	Anyone, holders of invitations, or only those who manage accounts; see Accounts.
Site name	none	What pages call the site, in place of the name the configuration gives.
Contact link	none	A web address, or mailto: and an email address, that every page's footer links.
Description, Keywords	none	The description and keywords meta tags, which search engines may show.
Posts per page	48	How many posts each page of a listing shows, from 1 to 500.
Tags beside a listing	24	How many of the page's most common tags show beside its posts, up to 500; none at 0.
Tag counts	on	Whether the tags beside a listing's posts and a post's tags show how many posts have each.
Left-out tags	tagme*	Tags that the tags beside a listing leave out, as Shimmie2's omitted tags, separated by spaces; * matches any characters.
Tag list minimum	1	How many posts a tag needs to appear in the tag lists, unless a list's mincount asks for another number.
Newest comments beside a listing	5	Up to 50; none at 0.
Comments per post in the comment list	10	From 1 to 100.
Wiki summaries	on	Whether a search for one tag starts with the first paragraph of its wiki page.
Conversion quality	80	How conversions to JPEG encode, from 1, the smallest files, to 100, the best looking.
Conversion background	#ffffff	The color that transparent pixels become in a conversion to JPEG.
Front page	The posts	What / shows: the posts, or the home page.
Site name in page titles	After the page's own title	Where page titles name the site: after the page's own title, before it, or nowhere, as Shimmie2's custom HTML headers choose.
Sitemap	on	Whether /sitemap.xml lists the site's pages; see Crawlers.
Image origins, Stylesheet and font origins, Embed origins	none	The other sites, such as https://www.youtube.com, that customization may load images, stylesheets and fonts, and embeds from; see Customization.

Ratings, comments, the wiki, tag categories, biographies, avatars, favorites, scores, parent and child posts, edit history, pools, notes, reports, and view counts are features, each on until an administrator turns it off. Favorites through reports came with migration 14, and view counts with migration 15, which sqoob runs when it starts, as it runs every migration it has not run yet. Ratings work differently from the rest: turned off, every post counts as safe, so whoever may see safe posts sees every post; see Ratings. Any other feature that is off leaves the site: its links and sections leave the pages, its own pages answer 404 Not Found saying that it is turned off, and searches and tag box terms that need it, such as comments>=3, charactertags>=2, or pool:Sketches, are refused. With tag categories off, tags show their whole names, in no color; with avatars or biographies off, profiles and comments show none. The feature's data stays as it was and comes back when the feature is turned on again; renaming a tag still takes its wiki page along while the wiki is off, and edits go on being recorded while edit history is off, so none are missing when it comes back.

A setting that the database holds in a form it cannot take, such as one edited by hand, has its default, with a warning in the log, so it never takes the site down.

Mail

SQOOB_MAIL_URL sends mail through a mail server on the same machine, or through a mail account's server:

Value	Sends through
smtp://localhost:25	A mail server on this machine, over plain SMTP.
sendmail or sendmail:/usr/sbin/sendmail	A local sendmail program, such as Postfix's.
smtps://user:password@mail.example.com	A mail account's server, over TLS on port 465.
smtp://user:password@mail.example.com:587?tls=required	A mail account's server, upgrading to TLS with STARTTLS.

A mail account sends through its SMTP server, which its provider lists next to the IMAP settings. Percent-encode special characters in the user name and password. sqoob refuses a URL that would send a password unencrypted, except to a server on the same machine. Check the configuration with sqoob mail test you@example.com.

Links in mail start from SQOOB_PUBLIC_URL, never from the host a request names, which anyone can forge. They lead to the /reset-password and /confirm-email pages, which fill in the code from the link. The code travels after the #, which browsers never send to a server, so it stays out of server logs; each message also carries it for typing in. Mail goes out from background jobs, ahead of uploads and other bulk work.

Web interface

The pages work without JavaScript in the browsers of the last two years: Chrome, Edge, Firefox, and Safari.

The interface uses a compact booru layout: search and tags on the left, uncropped thumbnails in the index, and numbered page links. The listing's tag sidebar uses only the current page's visible results; its counts are site-wide. Post information stays beside the image, with editing under “Edit post” and moderation under “Moderate.” On small screens, search stays at the top and a post's image comes before its tags and information. Tag lists can be collapsed. Account and administration pages share the same layout.

Page	Purpose
/posts	Posts matching the search in the search box, newest first unless the search sorts them otherwise, 48 to a page unless the site settings say otherwise. / leads here, unless the home page is the front page.
/home	The home page; see Customization.
/posts/edit	Changes the search's posts, or the ones chosen from it, all at once; see Bulk edits and Bulk Add.
/posts/{id}/replace	Replaces a post's file; see Files.
/posts/{id}	A post: its image, its tags by category with how many posts have each, and its rating, uploader, time, source, size, and format, with links to the posts before and after it in the search it was opened from. Those who may change its tags, source, or rating do it here, and those who moderate, delete, or purge posts do that here too.
/posts/{id}/download	The post's file, saved under Shimmie2's name for it: the post's ID and tags.
/tags/map, /tags/alphabetic, /tags/popularity, /tags/categories	The tags in use: by size, in alphabetical order, most used first, and by category. /tags leads to the map.
/help/search	Every search term, with examples. The search box's help links here.
/help/formatting	How to write wiki pages and comments, with each example as it shows.
/comments	The posts with the newest comments, ten to a page, each with its newest ten, or as many as the site settings say, and a form to reply.
/accounts, /accounts/{name}	The account list, and each account's profile, with its comments at /accounts/{name}/comments. The masthead's name leads to your own.
/wiki, /wiki/{title}	The wiki: its pages, and each page with its history, the changes between revisions, and its editor, which previews before saving.
/posts/{id}/history, /history	A post's edit history, and the newest edits of every post or of one account, with buttons to revert. Moderators also find there a form that reverts what an account changed since a day.
/popular/day, /popular/month, /popular/year	The posts of a day, a month, or a year with the highest scores.
/pools, /pools/{id}, /pools/changes	The pools, each pool with its posts in order and its editing, and the changes to pools' posts.
/notes, /notes/requests, /notes/changes	The posts with notes, the posts whose notes were asked for, and the newest changes to notes.
/upload	Uploads an image with tags and a source, and a rating for those who may give one.
/uploads/{id}	Follows an upload: it reloads itself while the file is processed, then goes to the new post, or says why the file failed.
/sign-in, /register, /forgot-password, /reset-password, /confirm-email	Signing in, registering, and the codes that mail carries.
/account	The signed-in account's email address, password, rating preferences, sessions, API tokens, and invitations.
/admin	Administration, for those who manage or moderate something: accounts, roles, tag rules, tag categories, and post counts, deleted posts, reports, files, Bulk Add, customization and blocks, the site's settings, background jobs, and audit records.
/jobs/{id}	A background job, for whoever may see it. It reloads itself until the job is done. Work done in parts goes on from each part to the next, and the page moves on to the last part that is done, so it shows how far the work has got.

The browser script only improves the pages. It suggests tags as they are typed, most used first and in their categories' colors, in the search box and every tag field; the search box keeps a word's - or ~. It shows the edit form's popular tags, which add or remove themselves with a click. It shows a note when the pointer rests on its place on the image, and lets a note's place be drawn on the image. It also fills in the code from a mailed link, and shows an upload's progress.

New password codes, API tokens, and invitations appear once, on pages that no cache may keep. The binary embeds the script and the stylesheet and serves them under /assets/, with names that change whenever their content does, so browsers keep them for good. Every page sends a Content Security Policy that runs only those, so nothing a user writes can run as a script in another's browser.

API

Route	Purpose
POST /graphql	The GraphQL API. Accepts only application/json bodies.
PUT /uploads/{id}	Receives the file for an upload opened with the beginUpload mutation.
GET /media/{key}	Serves stored files to those who may see their posts. Supports conditional and range requests.
GET /health	Returns ok.
GET /robots.txt, GET /sitemap.xml	The rules for crawlers, and the sitemap; see Crawlers.
GET /site.css, GET /favicon.ico	The custom stylesheet and the site icon; see Customization.

GraphQL and uploads identify the caller from the session cookie or an API token. Requests that could change something are refused when a browser sends them from another site. Errors from permission checks carry a code extension: UNAUTHENTICATED when signing in would help, and FORBIDDEN when it would not.

To upload, call beginUpload, send the file with PUT to the returned url, then call finishUpload with the upload's ID, its tags, and optionally its rating and source. Only the account that opened an upload can send its file and finish it. Finishing queues a job; query job until its state is SUCCEEDED, when it carries the new post, or FAILED, when it carries the reason.

Each tag keeps how many posts have it, whoever may see them, as postCount. tagSuggestions offers tags for autocomplete: those whose names begin with a prefix, most used first, and those with an alias that begins with it, or with no prefix the most used tags. tags lists the tags in use as the tag lists do, by name or most used first, from those whose names begin with startsWith and at least minCount posts have. tagAliases and tagImplications list the tag rules, and jobs lists background jobs for administrators.

Posts carry their comments and commentCount, and comments lists the newest; createComment, updateComment, deleteComment, and lockPostComments change them. wikiPage, wikiPages, and wikiHistory read the wiki, with each page's text as Markdown in body and as safe HTML in html. saveWikiPage takes the revision the text was edited from, basedOn, and refuses a save the page has moved past with the code CONFLICT.

Posts also carry the fields that the sections above describe: favoriteCount, favoritedBy, and favorited; score, myVote, and, for moderators, votes; parent and children; edits; pools; notes and noteRequestCount; and, for moderators, reports. setPostTags, createPost, and finishUpload take the tag box terms.

Anyone may read an account and list accounts, by name, narrowed by startsWith or role. An account's profile is in its fields: biography and biographyHtml, avatar, uploadedPostCount, commentCount, and comments, which count and list what your listings show. setBiography, setAvatar, and removeAvatar change your own profile, or, with account, the biography and avatar of an account you manage.

Anyone may read the site's settings with site, including which features are on and the customization. Administrators change them with setSiteSettings, which changes the settings it names, all of them or none, with setFeature, and with setRegistrationMode, and those who may customize the site change the customization with setCustomization, which says what sanitizing took out.

replacePostFile, convertPost, discardConversion, regenerateThumbnails, and regenerateAllThumbnails change posts' files, and bulkEdit starts a bulk edit; each returns the job that does the work. bulkAddPreview, startBulkAdd, bulkAdds, and bulkAdd add folders and report on them. Posts carry viewCount for those who may see it. While a feature is off, its fields, queries, and mutations fail with the code FEATURE_OFF, and so do searches that need it.

Counts change with every tag edit, in the same transaction, so they stay exact. After the database was changed by other means, such as by hand or from an old backup, recountTags or the button at /admin/tags checks every tag against its posts and corrects any count that is wrong. It needs MANAGE_TAGS, and runs as background jobs of 500 tags each: each job's next leads to the one after it, and the last job's recount says how many tags it checked and how many counts it corrected in all.

Bulk work runs as background jobs in small units: one job per file, tag rule changes update existing posts 100 at a time, and bulk edits change 100 posts at a time, or 10 when they work on files. The files of a Bulk Add wait behind all other work, so adding a big folder holds up nobody's uploads. Every limit applies to each file on its own, so a long batch never times out as a whole, and one bad file fails without stopping the rest. A job that panics fails on its own. A job the server stopped partway through starts again after a restart, up to three tries in all, so a job that keeps crashing the server cannot do so forever.

Testing

cargo test always tests SQLite. To include PostgreSQL 15 and MariaDB 10.11, start them in Docker and point the tests at them:

docker compose up -d --wait
export SQOOB_TEST_POSTGRES_URL=postgres://sqoob:sqoob@127.0.0.1:55432/postgres
export SQOOB_TEST_MARIADB_URL=mysql://root:sqoob@127.0.0.1:53306
cargo test

In PowerShell, set the variables with $env:SQOOB_TEST_POSTGRES_URL = "..." instead. npm run check type-checks the browser script.

Woodpecker CI runs the same checks on every push and pull request: cargo fmt --check, the browser script's type check and build, clippy with warnings as errors, and the full suite on all three databases. The pipeline is in .woodpecker.yaml; woodpecker-cli lint .woodpecker.yaml checks a change to it.