Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
183 changes: 120 additions & 63 deletions apps/typegpu-docs/src/content/docs/apis/textures.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -10,28 +10,27 @@ We assume that you are familiar with the following concepts:
- <a href="https://webgpufundamentals.org/webgpu/lessons/webgpu-storage-textures.html" target="_blank" rel="noopener noreferrer">Storage Textures</a>
:::

In a similar fashion to buffers, textures provide a way to store and manage data on the GPU. They allow for both read and write access from WGSL shaders, and can also be sampled in the case of sampled textures. The main advantage of using textures over buffers is their optimized memory layout for spatial data, which can lead to better performance in certain scenarios as well as additional functionality such as filtering and mipmapping.
Textures store spatial data on the GPU. They can be sampled, used as render targets, exposed as storage textures, and mipmapped.

TypeGPU textures serve as a wrapper that provides type safety and higher level utilities (such as automatic mipmap generation). They also allow - in a similar way to buffers - for fixed resource creation that can be used directly in shaders without the need for manual bind group management.
TypeGPU wraps WebGPU textures with typed views, usage checks, and helpers for common writes.

Let's look at an example of creating and using a typed texture.

```ts twoslash
import tgpu, { d } from 'typegpu';
import tgpu from 'typegpu';

const root = await tgpu.init();

const texture = root.createTexture({
size: [256, 256],
format: 'rgba8unorm' as const,
}).$usage('sampled');

const response = await fetch('path/to/image.png');
const blob = await response.blob();
const image = await createImageBitmap(blob);
const imageBitmap = await createImageBitmap(blob);

// Uploading image data to the texture (will be resampled if sizes differ)
texture.write(image);
const texture = root.createTexture({
size: [imageBitmap.width, imageBitmap.height],
format: 'rgba8unorm' as const,
}).$usage('sampled', 'render');

texture.write(imageBitmap);

// Creating a view to use in shader
const sampledView = texture.createView();
Expand Down Expand Up @@ -80,7 +79,7 @@ const texture = root.createTexture({
})
.$usage('sampled') // Can be sampled in shaders
.$usage('storage') // Can be written or read to as storage texture
.$usage('render'); // Can be used as a render target
.$usage('render'); // Can be used as a render target or image upload target
```

You can also add multiple flags at once:
Expand All @@ -97,77 +96,113 @@ const texture = root.createTexture({

## Writing to a texture

The `.write()` method provides multiple overloads for different data sources:
Most texture writes are one of: decoded images, blobs, image arrays, raw bytes, regions, or channel writes.

| Source | Use when |
| --- | --- |
| `texture.write(imageBitmap)` | You already have a decoded image with the right size. |
| `texture.write(imageBitmap, { resize: true })` | The decoded image should be resized into the texture. |
| `texture.writeAsync({ source: blob, ... })` | You have a fetched `Blob` and do not need to keep the decoded bitmap. |
| `texture.write([layer0, layer1])` | You want to fill a texture array or 3D texture one layer at a time. |
| `texture.write(bytes)` | You already have raw texel bytes. |
| `texture.write({ source, ... })` | You need a crop, destination region, or channel write. |

Image writes require `.$usage('render')`. Source and destination sizes must match unless you pass `resize: true`.

### Exact-size image upload

```ts twoslash
import tgpu from 'typegpu';
const root = await tgpu.init();
const response = await fetch('path/to/image.png');
const blob = await response.blob();
const imageBitmap = await createImageBitmap(blob);
// ---cut---
const texture = root.createTexture({
size: [imageBitmap.width, imageBitmap.height],
format: 'rgba8unorm',
}).$usage('sampled', 'render');

texture.write(imageBitmap);
```

:::caution[Browser support]
`GPUCopyExternalImageSource` also includes canvas, video, `ImageData`, `HTMLImageElement`, and `VideoFrame`, but browser support varies. Decode fetched images to `ImageBitmap` for portable code.
:::

### Resizing decoded images

```ts
// Image sources (single or array)
write(source: ExternalImageSource | ExternalImageSource[]): void
const texture = root.createTexture({
size: [512, 512],
format: 'rgba8unorm',
}).$usage('sampled', 'render');

texture.write(imageBitmap, { resize: true });

// Raw binary data with optional mip level
write(source: ArrayBuffer | TypedArray | DataView, mipLevel?: number): void
texture.write({
source: imageBitmap,
sourceOrigin: [16, 16],
sourceSize: [128, 128],
size: [256, 256],
resize: true,
});
```

### Writing image data
:::note
Resizing decoded images uses TypeGPU's render-pass blit path, so the destination must be a 2D texture with a renderable, float-sampleable format.
:::

### Blob helper

You can write various image sources to textures. `ExternalImageSource` includes:
- `HTMLCanvasElement`
- `HTMLImageElement`
- `HTMLVideoElement`
- `ImageBitmap`
- `ImageData`
- `OffscreenCanvas`
- `VideoFrame`
Use `writeAsync` when you have a `Blob` and do not need to keep the decoded `ImageBitmap`. With `resize: true`, TypeGPU passes resize options to `createImageBitmap` before uploading.

```ts twoslash
import tgpu from 'typegpu';
const root = await tgpu.init();
// ---cut---
const texture = root.createTexture({
size: [256, 256],
size: [512, 512],
format: 'rgba8unorm',
}).$usage('sampled');
}).$usage('sampled', 'render');

// From an ImageBitmap
const response = await fetch('path/to/image.png');
const blob = await response.blob();
const imageBitmap = await createImageBitmap(blob);
texture.write(imageBitmap);
const blob = await (await fetch('path/to/image.png')).blob();

// From an HTMLCanvasElement
const canvas = document.createElement('canvas');
const ctx = canvas.getContext('2d');
// ... draw on canvas
texture.write(canvas);
await texture.writeAsync({
source: blob,
size: [512, 512],
resize: true,
});
```

:::tip
If image dimensions don't match the texture size, the image will be automatically resampled to fit (requires 'render' usage).
:::
The resulting bitmap is still written as an image source, so the texture needs `.$usage('render')`.

### Writing arrays of images
### Texture arrays and 3D textures

For 3D textures or texture arrays, you can write multiple images:
Pass an array of images to write one image per layer.

```ts twoslash
import tgpu from 'typegpu';
const root = await tgpu.init();
declare const imageBitmap1: ImageBitmap;
declare const imageBitmap2: ImageBitmap;
declare const imageBitmap3: ImageBitmap;
declare const layer0: ImageBitmap;
declare const layer1: ImageBitmap;
declare const layer2: ImageBitmap;
// ---cut---
const texture3d = root.createTexture({
const texture = root.createTexture({
size: [256, 256, 3],
format: 'rgba8unorm',
dimension: '3d',
}).$usage('sampled');
}).$usage('sampled', 'render');

// Write array of images for each layer
texture3d.write([imageBitmap1, imageBitmap2, imageBitmap3]);
texture.write([layer0, layer1, layer2]);
```

Each image must match the layer size unless you pass `{ resize: true }`.

For 3D textures, add `dimension: '3d'`; the same `write([layer0, layer1, ...])` form writes depth slices.

### Writing raw binary data

You can write raw binary data directly to textures using `ArrayBuffer`, typed arrays, or `DataView`:
Use raw bytes when you already have texel data in CPU memory.

```ts twoslash
import tgpu from 'typegpu';
Expand All @@ -176,6 +211,7 @@ const root = await tgpu.init();
const texture = root.createTexture({
size: [2, 2],
format: 'rgba8unorm',
mipLevelCount: 2,
}).$usage('sampled');

// Using Uint8Array for RGBA data (4 pixels, 4 bytes each)
Expand All @@ -188,29 +224,48 @@ const data = new Uint8Array([
texture.write(data);

// Write to a specific mip level
const mipData = new Uint8Array(4 * 128 * 128); // Data for 128x128
const mipData = new Uint8Array(4); // Data for 1 pixel
texture.write(mipData, 1); // Write to mip level 1
```

You can also copy from another texture:
Raw writes do not need `.$usage('render')`, but the byte length must exactly match the texture format, size, and mip level.

### Regions and channel writes

Use an object descriptor for source crops, destination regions, or selected channels.

```ts twoslash
import tgpu from 'typegpu';
const root = await tgpu.init();
declare const imageBitmap: ImageBitmap;
declare const roughnessMap: ImageBitmap;
declare const metalnessMap: ImageBitmap;
declare const maskMap: ImageBitmap;
// ---cut---
const sourceTexture = root.createTexture({
size: [256, 256],
const texture = root.createTexture({
size: [512, 512],
format: 'rgba8unorm',
}).$usage('sampled');
}).$usage('sampled', 'render');

const targetTexture = root.createTexture({
size: [256, 256],
format: 'rgba8unorm',
}).$usage('sampled');
texture.write({
source: imageBitmap,
sourceOrigin: [16, 16],
sourceSize: [128, 128],
origin: [128, 64],
size: [128, 128],
});

targetTexture.copyFrom(sourceTexture);
texture.write({
channels: {
r: roughnessMap,
g: metalnessMap,
a: { source: maskMap, from: 'r' },
},
});
```

Region and channel writes follow the same render usage and resize rules as other image writes.

### Mipmaps

TypeGPU provides automatic mipmap generation for textures:
Expand All @@ -234,6 +289,8 @@ texture.generateMipmaps(); // Generate all mip levels automatically
The `generateMipmaps()` method requires both `'sampled'` and `'render'` usage flags, as TypeGPU runs a downsampling pipeline behind the scenes to generate the mip levels.
:::

You can also copy between equal-sized, same-format textures with `targetTexture.copyFrom(sourceTexture)`.

## Texture views

To create a view - which will also serve as fixed texture usage - you can use one of the available [texture schemas](/TypeGPU/apis/data-schemas/#textures). You can pass it to the `.createView` method of the texture.
Expand All @@ -254,7 +311,7 @@ const sampledView = texture.createView(d.texture2d(d.f32));
```

:::tip
If type information is available the view schema will be staticly checked against the texture properties.
If type information is available the view schema will be statically checked against the texture properties.

```ts twoslash
import tgpu, { d } from 'typegpu';
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ const carSpriteTexture = root
format: 'rgba8unorm',
})
.$usage('sampled', 'render');
carSpriteTexture.write(carBitmap);
carSpriteTexture.write(carBitmap, { resize: true });
const carSpriteView = carSpriteTexture.createView();

const linearSampler = root.createSampler({
Expand Down
13 changes: 2 additions & 11 deletions apps/typegpu-docs/src/examples/simulation/stable-fluid/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@ import { defineControls } from '../../common/defineControls.ts';

// Initialize
const root = await tgpu.init();
const device = root.device;

// Setup canvas
const canvas = document.querySelector('canvas') as HTMLCanvasElement;
Expand Down Expand Up @@ -86,20 +85,12 @@ let brushState: BrushState = {

// Load and create background texture
const response = await fetch('/TypeGPU/plums.jpg');
const plums = await createImageBitmap(await response.blob(), {
resizeWidth: p.N,
resizeHeight: p.N,
resizeQuality: 'high',
});
const plums = await response.blob();

const backgroundTexture = root
.createTexture({ size: [p.N, p.N], format: 'rgba8unorm' })
.$usage('sampled', 'render');
device.queue.copyExternalImageToTexture(
{ source: plums },
{ texture: root.unwrap(backgroundTexture) },
{ width: p.N, height: p.N, depthOrArrayLayers: 1 },
);
await backgroundTexture.writeAsync({ source: plums, size: [p.N, p.N], resize: true });

// Create simulation textures
const velTex = [createField('velocity0'), createField('velocity1')];
Expand Down
Loading