Skip to main content

Best Practices for Large-Scale Connector Projects

Why a connector sync producing thousands of Work Units is usually better split into one project per language pair than combined into a single project, with the Work Unit math behind it.

A connector integration can generate a lot more translation work than a manually-created project — sometimes thousands of files, languages, and workflow steps in a single sync. This guide covers how that volume adds up, and why splitting a large connector rollout into one project per language pair is usually the better structure than combining everything into one.

💡 Who is this for? This guide is for Account Admin, who sets up connector configurations, and Project Manager, who plans how a large connector rollout is structured across projects.


What counts as "large-scale"

A Work Unit in wxrks is created for every combination of file, workflow step, and target language. A project with 5 files, 2 workflow steps (say, Translation and Review), and 5 target languages produces 5 × 2 × 5 = 50 Work Units — and that multiplication is exactly why a connector feeding many files to many languages can grow a lot faster than it looks at first glance.

As a practical rule of thumb from operating large connector rollouts, a single sync request that would produce over 5,000 Work Units is a good signal that you're in large-scale territory and should think about project structure before it becomes a problem, rather than after.


Why split projects by language pair

It's tempting to keep a connector's entire output in one project — one thing to look at, one place to manage. At large scale, wxrks' experience running these integrations points the other way: grouping projects by language pair (one project per target locale, rather than one project covering every locale) tends to hold up better as volume grows, for a few concrete reasons:

  • Problem isolation. Localization issues are frequently locale-specific — a parsing edge case, a segmentation quirk, a regex rule that only misbehaves for one language. Splitting by locale means a fix or a regex/processing rule can be applied to the one locale that needs it, instead of being constrained to something that has to work identically for every language in the project.

  • Risk containment. If something goes wrong in one locale — a bad file, a stuck workflow step, an unexpected format — it stays contained to that locale's project instead of putting the entire sync at risk.

  • Queuing and throughput. Processing 15,000 items ten times over is more flexible than processing 150,000 items at once — splitting the load lets locales be processed serially, in parallel, or opportunistically, rather than all going through a single queue together.

  • Automation flexibility. Different locales often warrant different automation rules or parameters over time. Separate projects make that straightforward; a single shared project forces every automation rule to apply uniformly across all locales.

  • Management and reporting. A project that's already split by locale is naturally easier to assign — different Project Managers can own different locales — and easier to report on, without needing to filter one large project down by language every time.

  • Long-term scalability. Work Units and messages both multiply with scale. Removing "every locale in one project" as a multiplying factor is one of the most direct ways to keep a connector rollout scalable as it grows.


Consolidated vs. split at a glance

One project for everything

One project per language pair

The whole project shares a single status

Status can be tracked per locale

One shared parsing/filtering/regex setup for every locale

Locale-specific parsing and processing rules

A single queue for all processing

Independent, flexible queuing per locale

Reports need to be filtered by locale within the project

Each project is already scoped to one locale

Automation rules apply uniformly across all locales

Automation can be tuned per locale

A single issue puts the entire sync at risk

Risk is contained to the affected locale


Applying this to your connector

In practice, this means configuring your connector so that each sync (or each Repository Configuration, in GitHub's case) is scoped to a single target language rather than your full language list. How you set target languages and CI tags varies by connector:

If you're also relying on multiple pulls landing in the same project over time, see How to Combine Connector Projects for exactly what has to match (Organizational Unit, CI tag, source language) for that to happen — the same CI tag mechanics apply whether you're combining pulls into one project or deliberately keeping them separate by locale.


Related articles

Did this answer your question?