Common questions

Anyone (academic, industry, independent) who can submit a prediction file conforming to the required format. Each person joins one team; each team works on one track.

Yes. The three tasks share a data resource and a metric panel but are scored separately, so a team can enter one, two, or all three.

Yes. Coding agents, recursive LLM systems, evolutionary search. The framework is not the point; the absence of a human in the loop is. A person may write the starting prompt, but nobody may read intermediate results and steer the agent on the strength of them. Prizes require the trajectory, the prompts and the harness code. At least one of them has to be attached for an entry to enter the scoring queue at all, and anything we cannot fully verify is not eligible for a prize even once it has scored.

They are scored on the same tasks, the same metrics and the same hidden test sets, and their leaderboards sit side by side, but prizes are awarded per track, so a human-designed method never competes with an agent-designed one for the same prize.

A predicted set of cells for the target condition: expression for Task 1, expression plus 3D coordinates for Tasks 2 and 3. You never submit cell-type labels. The organisers assign types with a frozen classifier applied identically to every submission, so hidden labels are never exposed.

No. Nothing in the metric panel requires matching cell counts, and where a comparison would otherwise be confounded by count (the shape group), both point clouds are subsampled to a shared size first. Cell number is scored separately and explicitly, as count_log_ratio.

No. Every spatial metric is agnostic to translation, rotation and, outside one acknowledged blind spot, reflection. That blind spot is laterality: a mirrored embryo currently scores identically to a correct one.

Baseline implementations for each task, data-loading utilities, the exact evaluation code, example submission files, a standalone scorer for any .h5ad, documentation, and executed notebook tutorials.

Training data is released for method development. Validation and test ground truth are withheld while they can still affect the board, until the final phase a validation submission returns a leaderboard score, not the answers. At the start of the final phase the validation answers are released for every task, since ranking has moved to the hidden test split by then, and the test leaderboard opens. Test ground truth is never distributed.

Competition datasets are provided for research and educational purposes. Any use, reuse, publication, presentation, or redistribution of the data outside the Challenge must appropriately acknowledge the Virtual Embryo Challenge and credit Dr. Neil Chi’s group at the University of California, San Diego (UCSD), which generated the dataset. As the dataset has not yet been published, any use of the data in a publication or public presentation must receive prior approval from Dr. Neil Chi and the data-generating team. Any publication using the dataset must also appropriately credit Dr. Qingquan Zhang and Dr. Neil Chi.

You may train on external data, public or your own, and for Task 1 this is expressly encouraged. What you may not use is measured data from a held-out stage or genotype, by any route: not directly, not through a model pre-trained on it, and not through a public dataset that contains it. Those are E10.5 and E12.5 for Task 1; E7.5 and E7.75 in the embryo setting and E8.5, E10.5 and E12.5 in the heart setting for Task 2; and the Gata4 and β-catenin knockouts at E8.75 for Task 3. Nearness counts, and the stage label is not the test: external data is treated as a held-out stage when it sits closer to that stage than to the nearest stage we released for the same task, the midpoint being the boundary. Training on an external E9.45 embryo while E9.5 is held out is using the answer under a different name, and so is a dataset labelled by somite count or Theiler stage that lands in the same window. For the Task 1 extrapolation targets the window is stated outright rather than derived, and it relaxes the midpoint rule rather than following it: external data is excluded from after E9.5 up to and including E13.5, E10.5 and E12.5 are excluded absolutely, and anything after E13.5 may be used provided its source and stages are disclosed. The two ends of that window differ because one of them is ours to publish and the other is not. Data measured at exactly E9.5 is permitted, since E9.5 is a released training stage and data at a stage everyone already holds tells nobody anything they were not given. Data measured at exactly E13.5 is not, since E13.5 is the far edge of the protected window rather than the first point outside it, and it sits one day from the test target. For interpolation targets the boundary is the released stage on either side of the held-out one rather than a midpoint: external data may not be used if it stages strictly between the two released stages that bracket a held-out one. In Task 2 that means heart data strictly between E8.25 and E8.75, and embryo data strictly between E7.25 and E8.0. The bracketing stages themselves are permitted, being released training stages. The same applies to genotype: data from the held-out knockouts, from another allele of the same gene at a comparable stage, or from a perturbation that phenocopies them, is treated as the held-out condition. Every external source must be disclosed with the submission; an undisclosed one is a violation whether or not it changed the result, because the disclosure is what makes this checkable.

Hidden splits are defined by developmental stage, perturbation condition, or combinations thereof. Distinct from public splits. Models must learn generalisable developmental dynamics, not memorise.

Two anchors published with every score. The floor is copy_last (predict the preceding stage verbatim (wt_identity for Task 3). The ceiling is half the target scored against its other, disjoint half) deliberately not the target resubmitted verbatim, which is a number no honest generative model can reach.

A metric your output cannot produce, or that comes back NaN, counts as zero skill within its group. It is never treated as an excused absence.

Every attack found during the adversarial review is kept as a control row that is re-scored on every run. If a future change reopens one of those holes, the control row scores above its floor and the scoreboard flags it: the regression shows up in the table instead of needing another audit to find.

All ground-truth labels are hidden. Submission counts are limited in the final phase to reduce leaderboard probing. Duplicate registrations, unauthorised data use, code sharing across teams, or falsified results are grounds for disqualification.

Slack is the fastest, and the answer is visible to everyone else who has the same question. The Slack link below is an invitation to the Pantheon workspace. Accept it, then open #virtual-embryo-challenge, which is where the competition is discussed. Discord is the same conversation for people who prefer it, and needs no invitation. Email the organisers for anything that should not be public: eligibility, an entry you want withdrawn, a suspected scoring error, or anything involving your team’s data.

Email the organisers with your team name and the submission id: the id is on the submission’s own page, and it is what lets us find the exact file and the exact scoring run. Rescoring is possible while the phase is open; after a phase closes the board is fixed, so a problem is worth reporting the day you see it rather than at the end.

Both are on the site and both are accepted separately at registration: the Rules govern the competition (eligibility, teams, submission limits, prizes, what disqualifies an entry) and the Terms govern use of the site and the data, including the acknowledgement and prior-approval conditions on the datasets. The version you accepted is recorded against your registration, so a later revision does not silently change what you agreed to.

Yes. Open an issue on the veckit repository. Problems with the metric implementations or the scorer belong there rather than in email, because the fix is a commit other participants need to see.

Announcements go to Slack and Discord first, and to the address you registered with if you opted into news at registration. The timeline page is the authoritative schedule and is updated when a date moves.