# Inside DataHub: From Annotation Geometry to Reproducible Model Workflows Author: AlgoFly Team Published: 2026-11-12T13:08:01.271631+00:00 Canonical: https://algofly.ai/blog/datahub DataHub is AlgoFly’s computer vision data platform for teams that need to move from images to reviewed annotations, reproducible datasets and model experiments. Annotators work in an image editor; dataset owners organize projects and versions; engineers use the Python SDK and connected workers to consume the same data. The important connection is that an annotation remains meaningful after it leaves the canvas. A box drawn over a coin looks simple. Underneath it are coordinate systems, image revisions, class definitions, saved state and export contracts. Rotate the image, collaborate with another annotator or train against a frozen dataset, and those relationships become the real engineering problem. This article follows the implementation through those transitions, including the boundaries that keep a convenient interface from silently changing the meaning of the data. ![DataHub annotation workspace with a real coin image and annotation controls](/content/product-stories/datahub/overview.webp) The annotation workspace connects image\-relative geometry with class and review controls. Actual application interface. ## An annotation is data before it is a shape The editor supports rectangles, rotated boxes, polygons, points, lines, masks and multipart geometry, alongside pose keypoints, image classification and OCR representations. These types do not all reduce to the same object. A point has a location; a line has ordered geometry; a mask can contain regions that a single outline cannot preserve. Image\-level text and classification describe the image without needing a drawable boundary. The implementation therefore carries both a rendered representation and an image\-relative payload. Canvas zoom and pan belong to the viewing experience. They must not leak into exported coordinates. The annotation writer normalizes incoming payloads for the current image, deduplicates objects, validates the combined result and removes obsolete geometry fields when an existing object changes representation. Keeping an old angle or point list beside a new rectangle would create two competing descriptions of the same object. This matters particularly for rotated rectangles and transformed boxes. An enclosing axis\-aligned rectangle is not interchangeable with the original oriented geometry. The export layer makes format\-specific decisions rather than forcing every source object into a universal box representation. The same principle applies to polygons with multiple parts or holes: source detail should survive until the selected consumer requires a narrower representation. Geometry is also bounded by the image. Rotation, flipping, editing and restoration need to keep coordinates finite and associated with the correct dimensions. Non\-square images expose mistakes that square test images can hide: swapping width and height is a real coordinate transformation, not a cosmetic canvas adjustment. ## Transform pixels and labels together Image operations are a persistence problem as well as a geometry problem. The source\-image save path checks the image revision and source\-write sequence supplied by the editor. It reads the current source, verifies a stored checksum when present, prepares transformed pixels and a thumbnail, and validates the annotation payload that will accompany them. A successful visual preview alone does not establish that the saved source and labels agree. The current implementation records a recoverable transform intent with source and thumbnail checksums, dimensions, the next sequence, expected previous source fields and the annotations being published. Objects are staged before the source change is admitted. The commit path can recognize an already acknowledged operation and help recover an interrupted publication. If a previous transform has been recovered, the next writer is asked to reload rather than continuing from an uncertain image state. That is useful engineering in an ordinary interaction: rotating an image should not leave a new thumbnail pointing at old pixels, or new pixels paired with pre\-rotation coordinates. The code also marks dependent search indexing dirty and carries revision information into the response. Downstream readers have a way to distinguish the new image from its predecessor. Undo and redo use exported annotation snapshots, not just a screenshot of the canvas. The editor seeds its saved baseline from imported canvas geometry, avoiding a phantom first undo caused by rendering normalization. Image\-level changes also have to account for history: restoring coordinates from an earlier orientation would be incorrect even if the restored object looks plausible. The implementation’s source\-conflict handling preserves a draft and blocks saving against changed pixels. This is a deliberate boundary around history, rather than a promise that arbitrary source revisions can be undone indefinitely. ## Reuse an embedding, refine the prompt Assisted segmentation separates the expensive image encoding from the smaller prompt\-driven prediction step. The SAM service creates an image encoding containing features and original dimensions. Subsequent positive or negative point prompts, or a box prompt, can use that encoding to propose a mask. The model wrapper restores the cached features into an image predictor and requests a single mask result. The cache has an explicit lifecycle. Embeddings are serialized with format metadata, can use reduced\-precision storage and gzip compression, and are restored to the model’s device and floating\-point type when read. Concurrent creation is coordinated with an embedding lock. Stored readiness and eviction state distinguish a reusable embedding from one that needs to be produced again; workspace cache limits make retention a bounded resource. Prompt coordinates have to refer to the same image geometry as the encoding. The editor includes SAM\-specific geometry offsets, while source\-image revisions and dependent\-data invalidation matter whenever the pixels change. Reusing an encoding is valuable only while its relationship to the source remains correct. A proposed mask is still annotation assistance. For polygon output, the current contour conversion selects the largest external contour and simplifies it according to a complexity setting. That is a concrete tradeoff: fewer vertices are easier to edit, but a polygon proposal is not a lossless description of every disconnected mask region or interior hole. Teams should select a representation that fits their task and review the proposal before treating it as ground truth. ## Live previews are not committed labels Collaboration has two different jobs: show another person’s activity quickly, and keep the stored annotation state understandable. DataHub separates presence, cursors, selections and temporary annotation previews from committed updates. Socket handlers authorize the image room and check the sender’s access. The client checks project, image and room identity before applying incoming activity. Source stamps carry the image revision, source\-write sequence and annotation revision. Older snapshots can be ignored, and updates from another image source take a separate conflict path. A remote preview expires; it does not quietly become the editor’s canonical payload, undo baseline or saved annotation state. During a local drawing gesture, the preview handler also avoids applying a competing visual update. The save path uses conditional database updates rather than assuming that the state read earlier is still current. It checks source and annotation revision conditions, increments the annotation revision and returns the new source information. A changed revision can produce a conflict response. Source publication in progress also blocks an unrelated annotation write. On the client, changed source pixels preserve the user’s draft and require reopening the image before saving. These mechanisms should not be described as an automatic merge of every simultaneous edit. The implementation provides fast shared context, stale\-update detection and explicit recovery boundaries. That is a more useful contract than allowing the last network response to define which coordinate system or source image a team intended. ## Freeze a dataset, including intentional absences The working project is an editable collection. A dataset version is a materialized training input with selected images, split configuration, classes, preprocessing and augmentation settings. Creation resolves eligible image selection and checks source stability before a background processing run produces outputs. The UI can report processing progress rather than making the browser wait for every image transformation. One subtle distinction is whether an empty annotation means “reviewed, nothing here” or “nobody has labeled this yet.” The writer records an explicitly saved empty annotation as a negative example. Version eligibility includes positive annotations and these intentional negatives, while simply opening an image does not make it training\-ready. The current mixed supervised build requires at least one positive image; negative examples can accompany it. Materialized annotation snapshots retain the original annotation count and a source\-negative flag before class filtering. This prevents a filtered\-empty image from being silently promoted to a confirmed negative. Removing all objects because a version excludes their classes is a different operation from an annotator asserting that an image contains no target objects. Builds retain processing\-run identity, and generated snapshots carry image dimensions, class metadata and preprocessing history. Split assignment and augmentation use version\-related seeds in their processing paths. Completed version outputs are the reproducible handoff: later work in the project should lead to a new build when a new training input is wanted. A queued build should not be confused with an already completed immutable artifact; its processing state still matters. ![DataHub project image grid with annotation and dataset organization context](/content/product-stories/datahub/detail.webp) Project browsing is the editable working collection; dataset versions provide the materialized handoff to downstream tools. ## An export is a geometry contract The export registry connects named formats and aliases to dedicated exporters. It includes YOLO detection, segmentation and oriented boxes; COCO and COCO RLE; pose formats; LabelMe; semantic PNG masks; classification manifests; OCR outputs and other consumer\-specific layouts. These are different schemas, not alternate filenames for identical labels. A segmentation mask, oriented box and OCR region need different preservation rules. A semantic PNG also needs a class\-to\-pixel interpretation, while a pose export needs keypoint structure. The platform’s native geometry remains richer than some targets. Choosing a format is therefore an explicit conversion decision, and stricter consumers should reject incompatible input rather than quietly claiming full fidelity. Generated exports have their own lifecycle, separate from version creation. The implementation tracks attempts, publication and recovery, rather than treating a background ZIP task as proof that a downloadable artifact exists. The SDK exposes an uncertain\-publication recovery state and does not automatically start another generation when the first result is ambiguous. Export manifests are designed to be replayed without requiring the entire entry list in memory. A disk\-backed SQLite sequence stores ordered entries and enforces unique archive names. Writes consult a resource budget, and large manifests have a streaming path. This is the kind of infrastructure that becomes important when a simple dataset download grows into many image and label objects. Interactive browsing has a different storage concern: avoiding repeated downloads without showing stale pixels. The annotator maintains bounded caches for images, decoded sizes, annotations and project pages. Image cache keys include image identity and revision; source URLs carry revision information. A changed source can invalidate dependent entries while normal navigation continues to reuse cached content. Search indexing is also marked pending when labels change, so the stored annotation and its derived search representation have an explicit synchronization path. ## Use the same version from Python The Python SDK follows a workspace → project → version object model. A script can resolve an existing version and request a consumer format without reproducing browser\-side dataset logic. The following example assumes that the SDK client is already configured for an authorized deployment; identifiers are illustrative. ``` from algofly import AlgoFly client = AlgoFly() project = client.workspace("workspace-id").project("project-id") version = project.version(3) archive = version.download("COCO", location="./datasets") ``` The download call can generate or reuse an export, wait for readiness and download the resulting artifact. Scripts still need to handle failures, access checks and recovery\-required states. The CLI additionally provides dataset upload discovery, format hints, split inference, tags, batch names and a dry\-run option. It accepts images, directories and supported archive inputs. The current CLI is an ingestion interface; it should not be presented as a command for every operation exposed by the Python SDK. ## Run experiments on registered workers Training connects a project version and export to a selected trainer and worker. Worker registration has explicit project\-sharing scope and a separate choice about allowing custom jobs. Hardware reports and heartbeats inform admission. Disk assessment and resource checks help determine whether a job can be prepared and executed, rather than assuming every reachable host can run every workload. Submission includes an idempotency key. The server fingerprints the request and uses that identity to recognize a repeated submission; reusing a key for a different request is a conflict. A worker then claims a run through a fenced lifecycle. Claim tokens and epochs protect subsequent status and finish operations against a stale worker continuing after ownership changes. The SDK exposes that lifecycle directly. Trainer identifiers, parameters and worker eligibility should come from the configured platform rather than a hard\-coded assumption about available hardware: ``` run = client.training.start( project_id=project.id, version_id=version.id, export_key="ready-export-key", trainer_id="configured-trainer-id", worker_id="registered-worker-id", parameters={}, idempotency_key="experiment-001", ) completed = client.training.wait(run["_id"]) ``` The returned run identity connects submission to polling, events and artifacts. Output uploads calculate SHA\-256 checksums and derive artifact identity from the run, filename and content checksum, allowing acknowledged uploads to be recognized again. Progress remains inspectable beyond the initial request. ## Keep the model’s history attached Conversion starts from an artifact identity, not just a file extension. Source checks inspect the completed run, adapter and artifact membership before selecting a conversion path. Validated parameters describe image shape, batch, precision and target format. Dynamic TensorRT requests require explicit minimum, optimum and maximum profiles with ordered dimensions; the optimum must agree with the selected batch and image size. Hardware support remains specific. Current SDK guidance restricts the general ONNX\-target conversion route to validated worker cohorts, and adapter\-specific paths have their own requirements. An ONNX file is not proof that every device, input size or precision mode has been validated. Converted artifacts need their source lineage and target constraints alongside the resulting bytes. Inference similarly begins with a source artifact and declared options. Evaluation goes further: it binds a model to a held\-out version and split, requires an explicit model contract and maps frozen dataset classes onto model output indices. The class validator requires a complete bijection for the selected class set. Alphabetical label order is not accepted as a substitute for model identity. The current native evaluation contract supports classification and axis\-aligned detection under defined limits. Unsupported crowd or ignore semantics, rotated boxes and ambiguous geometry are rejected in the detection path. Empty ground truth must be an explicit saved negative. Cohort selection can be seeded and bounded; uploaded ONNX contracts remain user\-declared and do not acquire invented training provenance. That is the through\-line of DataHub’s implementation: preserve enough information to explain what was edited, what was built and what was run. The platform combines annotation craft with the storage, coordination and validation work needed to carry those decisions into a model workflow. Explore [DataHub’s product overview](/products/datahub), or [discuss a computer vision data platform](/contact) with AlgoFly.