Simulated Color Separation in Screen Printing: The Definitive Technical Guide

Simulated color separation is the most versatile and widely used method for reproducing photographic and multi-color artwork in screen printing. Moreover, it is the technique that makes photographic t-shirt prints possible without the limitations of CMYK process printing — delivering vibrant, opaque color on dark garments, using spot-mixed inks for accurate color matching, and tolerating the registration and dot gain realities of production screen printing.

Furthermore, understanding simulated color separation — what it is, how it differs from other methods, and how to execute it at a professional level — is essential knowledge for any serious screen printer or prepress specialist.

What Is Simulated Color Separation?

Simulated color separation (also called simulated process, or “sims”) is a technique in which a full-color photographic image is reproduced using a set of spot-mixed opaque inks, each printed through a halftone screen. In contrast to CMYK process printing (which uses only four standard transparent inks), simulated process uses a palette chosen for the particular artwork — with no preset number of colors. The aim is to reproduce the artwork with as few inks as possible: since the colors overlap and mix optically, a palette of a few screens yields far more colors in the print than it has slots.

The term “simulated” refers to the fact that the process simulates the appearance of full-color printing without using the standard four-color CMYK model. In other words, the colors are “simulated” by mixing halftone dots of opaque spot colors rather than by overlaying transparent standard process inks.

The Key Distinction from CMYK Process

In CMYK process printing:

• Four fixed inks (Cyan, Magenta, Yellow, Black)
• Transparent inks — colors are produced by optical mixing of overlapping dots
• Limited to the CMYK gamut
• Works best on white substrates — on a dark shirt, process inks need an underbase and even then do not give colors as vivid as inks mixed to Pantone numbers

In contrast, in simulated color separation:

• Custom palette of inks mixed to specific Pantone numbers, with no preset number of slots
• Can work on dark garments with a white underbase
• Color gamut can be expanded beyond standard CMYK by choosing palette colors strategically
• The halftone angle and frequency are set the same for every color — that is the rule — to eliminate interference with the mesh threads; what matters is matching the mesh to the frequency and choosing the angle itself: 22.5° or 67.5°

Why Simulated Process Outperforms CMYK for T-Shirt Printing

CMYK process printing is an option for printing on a white shirt. For printing on dark garments, simulated process delivers better results, for several reasons:

1. Opaque inks work on dark substrates. Process inks on a dark garment give poorly saturated colors even over an underbase. Inks mixed to Pantone numbers, printed on a white underbase, give saturated, accurate color on a garment of any color.

2. Ink colors can be optimized for the specific artwork. CMYK is fixed — you always use the same four inks. In contrast, for a design dominated by earthy tones, a simulated process palette can include warm browns, tans and flesh tones that may be outside the CMYK gamut altogether. Similarly, for a design with vivid blues, the palette can include a royal blue or cerulean — a color taken straight from the Pantone guide reproduces more accurately than CMYK cyan.

3. More forgiving of dot gain. Opaque inks — standard plastisols — do mix where they overlap, but not to the degree process inks do. Consequently, shadow fill and color shift from dot gain are far less destructive.

4. Better registration tolerance. CMYK process requires tight registration (±0.1mm) for correct color. However, simulated process, because it uses opaque inks, is less sensitive — slight misregistration shifts an edge above all, and does not shift the color as strongly as it does with overlapping transparent CMYK dots.

The Simulated Color Separation Workflow

Step 1: Image Preparation

Begin with a high-resolution RGB image — 200–300 DPI at the intended print size. Specifically, simulated process separation requires:

• Correct color balance: The image must be color-accurate before separation begins. Wrong white balance in the source photo, or uncalibrated monitor display, will produce an inaccurate separation.
• Correct shadow and highlight range: Check levels — the darkest point in the image should be close to 0,0,0 (black) and the lightest point close to 255,255,255 (white). As a result, flat, low-contrast images produce flat, low-contrast prints.
• Sharpening: Apply unsharp mask before separation (Amount: 100–150%, Radius: 0.8–1.2px, Threshold: 2–4). This is because the halftone screen printing process slightly softens fine detail, and pre-sharpening compensates for that loss.

Step 2: Color Analysis and Palette Selection

Before creating any separations, analyze the image to determine which colors will be most effective in the palette. This is where simulated process requires creative judgment, not just technical execution.

Identify the dominant hues: What are the 3–5 most important colors in the image? For example, for a portrait: skin tone, hair color, background color, shadow color, highlight white. For a landscape: sky blue, green, earth tone, shadow, white highlight.

Identify the gamut extremes: What is the most saturated color? The darkest shadow? The brightest highlight? Consequently, these define the extremes the palette must reach.

Decide how many colors to use: the number follows from how many steps the ramp has in the particular image, and from the color of the garment — a dark shirt adds an underbase, which is one more screen. Budget (number of screens = number of inks = cost) and press capacity set the ceiling.

Step 3: Creating the Separations

Simulated process separations are created manually in Photoshop — this is the “manual color separation” that defines professional prepress work, as distinct from automated software solutions.

The core technique: each color channel in the separation is built by selecting and masking the tonal range that that particular ink will represent in the final print. In practice, this means building each channel independently through the following steps:

Underbase white (for dark garments): Built first. It uses luminosity/brightness information from the image to determine where white underbase is needed. Shadow areas (dark garment contributes shadow) get less or no underbase; highlight areas get full white. Typically, it is a halftone channel, not solid white.

Black (or dark shadow color): The shadow/detail channel. It provides depth, shadow detail, and outlines. Usually a standard black or very dark brown. There is one exception to the shared frequency: the black channel may be given a higher one — with the colors at 65 LPI, black goes at 90, for example. The black dot is the heaviest one visually: the eye catches it most easily, it makes visible graininess and it darkens the image. Raising the frequency makes the dot smaller, so black still darkens the drawing nicely but stops jumping out. The black dots then do not line up with the color halftone — some fall between them — and that does no harm. The angle is never changed.

Highlight white (for dark garments): A separate white channel printed last — on top of all other colors — to add brilliant pure-white highlights. Importantly, this is different from the underbase; this is the final white accent layer.

Color channels: Each color channel represents the tonal contribution of that ink to the final image. For example, the red channel, orange channel, flesh tone channel, or blue channel — each is extracted by:

1. Converting the image to Lab color mode to isolate luminance from chroma
2. Using Selective Color or Hue/Saturation to isolate the target color range
3. Creating a grayscale representation of that color’s tonal contribution
4. Applying halftone screening at the appropriate frequency and angle

Step 4: Halftone Frequency and Angle Assignment

The angle: the same for every color. On fabric the threat is not interference between the colors, it is interference between the halftone and the mesh threads — so the angle is chosen against the threads, not against the other channels. The ones that work best are 22.5° and 67.5°: they sit furthest from warp and weft (0° and 90°) and furthest from the 45° diagonal, that is, from every direction in which the weave has a regularity of its own.

The frequency: usually 45–65 LPI. In particular cases — an experienced printer and good equipment — 90 LPI can be printed. The frequency is shared by all the colors; the only exception is black, which may be raised (step 3). The angle is never changed.

The mesh is matched to the frequency: at least 4–4.5× the LPI, counted in threads per inch. For 45 LPI that comes to 180–200 TPI (71–79/cm), for 65 LPI 260–290 TPI (102–115/cm), and 90 LPI already calls for a mesh of the order of 355–405 TPI (140–160/cm).

Step 5: Dot Gain Compensation

Apply a dot gain compensation curve to each channel before film output. Simulated process is particularly sensitive to dot gain in the highlight areas — fine highlight dots that lose even 5% gain cause visible shadow fill in what should be bright, open tonal areas.

A standard starting curve for simulated process on standard garments (medium mesh) is:

• 10% input → 6% output
• 25% input → 18% output
• 50% input → 38% output
• 75% input → 63% output
• 90% input → 82% output

After measuring actual press dot gain, adjust these values accordingly.

Step 6: Proofing

The separation is built on channels, so the composite preview is there from the start — it does not have to be assembled separately. What is left is to compare it to the source image and to correct any channel that is too heavy (filling the shadows) or too light (losing highlight detail).

Beyond that it is worth making a halftoned proof, closer to what will actually come out on film.

A simulated print. Duplicate the separated artwork and convert the channels to a bitmap at the intended frequency and angle, then go through them one by one. Only then can you see exactly where the dots land — and things turn up that the channel preview does not show at all: 2–5% black in a piece of lettering where there should be no black, for instance. On a grayscale channel you will not notice it; on a halftoned bitmap every dot is visible.

A blur at the end. Once all the channels have been checked, select the spot channels of the separation and apply a Gaussian Blur at a radius of 1.5–1.7. The halftone dots spread slightly, so instead of a grid of single pixels you see something closer to how the ink will sit on the fabric.

Ink Selection for Simulated Process

The ink palette in simulated process is the creative core of the technique. Unlike CMYK (where you have no choice), simulated process allows custom palette selection that can be optimized for each image.

Building a Palette

Start with black and white. Nearly every simulated process job includes a black (or dark shadow) channel and a white (underbase and/or highlight white). Together, these provide the full tonal range of the image.

Add the primary hues needed by the image. For instance, a portrait needs skin tones. A landscape needs blues and greens. A product shot needs the product’s brand colors.

Use mixed colors, not standard gamut. Simulated process works best with custom-mixed inks that precisely match the dominant colors in the image — not standard primary colors. As a result, a flesh tone mixed specifically for the skin in the image will reproduce more accurately than trying to build flesh from standard red, yellow, and white halftone combinations.

Simulated Process vs. Index Color: When to Choose Each

Both simulated process and index color separation produce photographic results with opaque inks on dark garments. However, the choice between them depends on design content and production requirements.

Choose simulated process when:

• Print quality and color accuracy are the top priorities
• Press has reliable, precise registration
• Design contains complex skin tones or smooth gradients

On the other hand, choose index color when:

• Moiré is a recurring problem on press
• Design will be printed on textured fabrics
• Press registration is less precise
• A faster prepress workflow is needed
• Budget allows more screens — index color needs more of them than simulated process by design

Troubleshooting: Simulated Color Separation Problems

Problem 1: Print Looks Too Dark / Shadows Filled In

Symptoms: Shadows blocked solid, highlight detail present but mid-tones too dark overall.

Causes and fixes:

• Black channel too heavy: Reduce the opacity of the black separation channel; lighten the curve in shadow areas.
• Dot gain not compensated: Apply dot gain compensation to all channels; this is most critical for shadow areas where large dots are already close to merging.
• Underbase too solid under shadow areas: For dark garment work, reduce or eliminate underbase under the darkest shadow areas — the dark garment contributes shadow depth naturally.
• Too much ink overlap in mid-tones: Multiple channels contributing ink to the same mid-tone zone create cumulative opacity. As a result, reduce the midtone density of secondary channels.

Problem 2: Colors Look Washed Out / Low Saturation

Symptoms: Print looks desaturated, muddy, or dull compared to the original image.

Causes and fixes:

• Underbase insufficient: Colors printing on insufficient white underbase will be contaminated by the dark garment. Therefore, check underbase opacity first.
• Ink colors too neutral / not saturated: The mixed inks may not be saturated enough. Compare printed ink patches to the palette colors — if the printed ink looks muted, check mixing and ink quality.
• Process inks where standard ones belong: process inks over an underbase will always give a less saturated color than standard plastisols over the same underbase. If the color has to be vivid, print it with standard plastisols.
• Too much black channel: Excessive black reduces saturation of all colors. Therefore, reduce black channel density, particularly in saturated color areas.

Problem 3: Skin Tones Look Wrong

Symptoms: Flesh tones appear too orange, too pink, too red, or look posterized rather than smooth.

Causes and fixes:

• Flesh tone ink color inaccurate: Re-mix the flesh tone ink — compare printed ink patch to the target skin tone. Then adjust toward the correct hue.
• Insufficient flesh tone channel range: The flesh tone channel should carry a wide range from very light (highlight on skin) to medium (mid-tone skin). Therefore, check that the channel covers the full tonal range of the skin in the image.
• Black channel too heavy in skin areas: Black in skin tones creates a grayish cast. As a result, reduce black channel density in flesh areas using selective channel masking.
• Color profile mismatch: The monitor display is not accurately representing the image colors. Consequently, use a calibrated monitor and work in a consistent color space (sRGB for most photography).

Problem 4: Moiré Pattern Visible in Print

Symptoms: Repeating interference pattern visible in halftone areas.

Causes and fixes:

• Halftone frequency too close to the mesh count: the mesh must be at least 4–4.5× the frequency, counted in threads per inch — 180–200 TPI (71–79/cm) at 45 LPI, 260–290 TPI (102–115/cm) at 65 LPI. Below that ratio the two frequencies are so close that they start to beat against each other whatever the angle. Then either lower the frequency or take a finer mesh.
• The fabric’s own texture: interference also arises between the weave of the fabric itself and the halftone frequency, not only between the halftone and the mesh threads. If the mesh and the angle are right and moiré still shows up, look at the fabric.
• Misregistration causing channel interaction: Even slight misregistration between two halftone channels can create moiré in the overlap areas. Therefore, check and correct press registration.
• Consider switching to index color: For persistent moiré problems, index color separation eliminates the issue entirely.

Problem 5: Fine Dots Not Printing

Symptoms: very light tonal areas (3–10% dots) print as blank white, and gradients jump from white straight to the first visible dot instead of fading out smoothly. At the other end of the scale the symptom is the opposite: dots of around 98% merge into a solid patch and the shadow loses its detail.

These are two different faults and they have to be told apart:

• The fine dots cannot be washed out of the emulsion (highlights, 3–10%). With too long an exposure the light gets under the fine dots, the emulsion beneath them hardens and will not wash out — what is left is a solid patch of emulsion that lets nothing through. Shorten the exposure, work on yellow mesh and set the time with a step wedge.
• The fine dots will not hold on the screen (shadows, around 98%). Here it is not about washing the dot out but about holding the small islands of emulsion between the dots. When they fall out, a solid patch of ink goes through the screen and there is no detail left in the shadow.

Other causes:

• Mesh too coarse for the dot size: fine highlight dots require a finer mesh. Increase the mesh count.
• Dot gain compensation removing too much: if the curve reduces 10% input to 3% output, dots that small may not print reliably. Set the minimum dot in the RIP to 5–8%.

Common Mistakes in Simulated Color Separation

1. Starting with a poorly prepared image. Simulated process cannot compensate for a flat, low-contrast, low-resolution, or poorly color-balanced source image. Therefore, always correct the image before separating it — the better the source, the better the separation.

2. Fixing the color count before analyzing the image. A 4-color simulated process can look excellent for graphic work and still fail to reproduce complex photography convincingly — and the other way round: on a white shirt five colors can give a good portrait, because the lightest tone is the fabric itself. The count follows from the ramp in the particular image and from the color of the garment, not from a figure set in advance.

3. A halftone frequency that does not match the mesh, or the wrong angle. Either one ends in interference. The mesh must be at least 4–4.5× the frequency, counted in threads per inch, and the angle — the same for every color — should sit well away from the directions of the weave: 22.5° or 67.5°. The angles used when printing on paper, where every color goes at a different angle, do not carry over to fabric.

4. Not building a halftone underbase for dark garment photographic work. A solid white underbase under a photographic simulated process print produces heavy, stiff prints with crushed shadow detail. Therefore, the halftone underbase is a more sophisticated but significantly better approach.

5. Relying on automated separation software without manual adjustment. Automatic separation software provides a starting point, not a finished separation. As a result, every automated separation should be reviewed and adjusted channel by channel — shadow balance, skin tone accuracy, and highlight preservation always require manual refinement.

6. Not proofing the separation before burning screens. A soft proof in Photoshop reveals problems before screens are burned. For example, discovering a major color issue after screens are made costs hours; discovering it during proofing costs minutes.

7. Using the same dot gain curve for every job. Dot gain varies with mesh count, ink type, garment, and press conditions. Therefore, build and apply a specific compensation curve for each press/mesh/ink/substrate combination.

Summary

Simulated color separation is the professional standard for photographic screen printing — it combines the opacity and vibrancy of spot-color printing with the tonal complexity of photographic reproduction. Furthermore, it requires more prepress skill than either CMYK process or index color separation, but it delivers results that neither of those methods can match for multi-color photographic work on dark garments.

In conclusion, the key to excellent simulated process work is in the details: precise image preparation, thoughtful palette selection, careful channel construction, correct halftone assignment, dot gain compensation, and rigorous proofing. Each of these steps contributes to the final print quality. Therefore, skip any one of them, and the print will show it.

Dragonfly Colors specializes in manual simulated color separation — handcrafted, channel-by-channel prepress for screen printers who demand the best possible result. Contact us to discuss your next photographic printing project.