Representations¶

A representation turns a sparse EventStream into a dense tensor (most are [C, H, W], ready for NumPy / PyTorch) or, for pset(), a sparse point array. Most are reachable three ways:

stream.voxel(bins=5)                       # method
ecv.voxel(stream, bins=5)                  # free function (OpenCV-style)
stream.flatten("voxel")                    # by name (defaults; see "By name" below)

The method and free-function forms take the representation’s parameters directly. The name form (a string like "voxel") selects a representation for flatten(), view(), open(..., repr=...), and reader.with_repr(...) — but only reader.with_repr(name, **opts) also forwards the representation’s parameters; the other three use its defaults. A few representations don’t fit all three forms; each is flagged in its section below.

Summary¶

Name	Method	Channels	Dtype	What it encodes
`polarity`	(none — default view, or by name)	2	`uint64`/`uint8`	Per-pixel event count, split into positive/negative planes. The default view when no representation is requested.
`binary`	(none — `flatten(binary=True)`, or by name)	1	`uint8`	Pixel fired at all (`1`) or not (`0`), both polarities merged.
`count`	`count()`	1	`uint64`/`uint8`	Total events per pixel, both polarities summed.
`voxel`	`voxel()`	`bins`	`float32`	Signed, time-interpolated polarity across `bins` temporal bins over the last `window_ms`.
`tsurf`	`tsurf()`	2	`float32`	Time surface: `exp(-age/tau_ms)` of the latest event per pixel/polarity.
`atsurf`	`atsurf()`	2	`float32`	Averaged (HATS-style) time surface: mean of `exp(-age/tau_ms)` over all events per pixel/polarity.
`tencode`	`tencode()`	3	`uint8`	Latest polarity + normalized age within `window_ms`, as an RGB-like image.
`mcts`	`mcts()`	10	`float32`	Multi-channel time surface: 5 log-spaced windows up to `max_window_ms`, per polarity.
`flow`	`optical_flow()`	2	`float32`	Dense Lucas-Kanade optical flow `(flow_x, flow_y)` on the time surface, pixels/ms.
`pset`	`pset()`	— (`[N, 4]`)	`float32`	Sparse, normalized `(x, y, t, p)` point array — not a dense frame.
`labels`	`connected_components()`	1	`uint64`	Connected-component labels of an existing frame (post-processing, not stream-level).

Two rows are special cases rather than string-selectable representations: pset returns a sparse point array (not a dense frame), and labels is computed from an existing frame — so neither is a valid repr= / flatten / view name. See their sections below.

The dtype column lists uint64/uint8 for the two representations with a normalize option (count and polarity): raw uint64 counts, or uint8 rescaled so the busiest pixel maps to 255 when normalize=True. The default differs by representation: polarity normalizes by default (True), while count returns raw counts by default (False) so the frame preserves exact event totals. The rest have no normalization option. view always auto-contrasts for display regardless of the frame’s dtype.

Polarity¶

stream.view()                  # implicit — polarity is the default when nothing is stored/named
stream.flatten("polarity")
ecv.flatten(stream, "polarity", normalize=False)

Two channels, [positive, negative]: raw event counts per pixel per polarity (uint64), or uint8 rescaled to the busiest pixel when normalize=True (the default). This is the frame rendered whenever no representation is requested and the stream has no stored repr (from open(repr=...)) — there is no dedicated .polarity() method, only the name form and the implicit default.

Binary¶

stream.flatten(binary=True)    # dedicated flag
stream.flatten("binary")       # equivalent, by name

Single channel, uint8, 1 where a pixel fired at all (either polarity) and 0 elsewhere. No parameters, and — like polarity — no dedicated method; only reachable via flatten(binary=True) or the "binary" name.

Count¶

stream.count()                    # raw uint64 counts (default)
ecv.count(stream, normalize=True) # uint8, busiest pixel → 255

normalize (bool, default False) — when True, rescale to uint8 (busiest pixel → 255) instead of the default raw uint64 counts.

Single channel: total events per pixel, both polarities summed. The plainest “how much happened here” frame — unlike polarity, it collapses polarity into one intensity map. The raw uint64 default keeps exact totals (e.g. count().numpy().sum() equals the event count), which is what you want for analysis; pass normalize=True for a display-scaled image.

Voxel grid¶

stream.voxel(bins=9, window_ms=30.0)
ecv.voxel(stream, bins=5, window_ms=50.0)

bins (int, default 9) — number of temporal bins; must be at least 1.
window_ms (float, default 30.0) — trailing time window before the stream’s last event.

bins channels, float32, signed. Each event contributes +1/-1 (by polarity), linearly interpolated between its two nearest bins by how far into window_ms it falls — events outside the window are dropped.

Time surface (`tsurf`)¶

stream.tsurf(tau_ms=30.0)
ecv.tsurf(stream, tau_ms=10.0)

tau_ms (float, default 30.0) — exponential decay constant; must be finite and positive.

Two channels, [positive, negative], float32. Each pixel/polarity holds exp(-age_ms / tau_ms) for its most recent event only (0 if that polarity never fired there) — a snapshot of “how recently did this fire.”

Averaged time surface (`atsurf`)¶

stream.atsurf(tau_ms=30.0)
ecv.atsurf(stream, tau_ms=30.0)

tau_ms (float, default 30.0) — exponential decay constant; must be finite and positive.

Two channels, [positive, negative], float32. Same exponential response as tsurf, but averaged over every event that hit the pixel (HATS-style) rather than keeping only the latest — recurring activity reads brighter than a single stale hit.

Tencode¶

stream.tencode(window_ms=30.0)
ecv.tencode(stream, window_ms=30.0)

window_ms (float, default 30.0) — trailing time window; must be finite and positive.

Three channels, [positive, age, negative], uint8. Per pixel: whichever polarity fired most recently sets its channel to 255 (the other stays 0), and the age channel holds that event’s age rescaled to 0-255 across window_ms. Reads like an RGB image (red/green-style polarity planes plus a brightness-coded age plane).

MCTS (multi-channel time surface)¶

stream.mcts(max_window_ms=30.0)
ecv.mcts(stream, max_window_ms=16.0)

max_window_ms (float, default 30.0) — largest window; must be at least 1.0.

Ten channels, float32: 5 log-spaced windows (from 1 ms up to max_window_ms) for each of negative/positive (channel names like negative_1.000ms … positive_30.000ms). Each channel holds max(1 - age/window) over events within that window — a multi-timescale generalization of tsurf.

Optical flow¶

stream.optical_flow(window=3)
ecv.optical_flow(stream, window=5)
stream.view("flow")             # by name; also valid for open(repr="flow")

window (int, default 3) — half-width of the least-squares neighbourhood (a (2·window+1)² patch); must be at least 1.

Two channels, [flow_x, flow_y], float32, in pixels/ms. Dense Lucas-Kanade on the time surface (the Surface of Active Events): fits one velocity per pixel by least squares, falling back to normal flow along the gradient in the aperture-problem case, and leaving flat, event-free windows at zero. Flow is emitted only at pixels that saw an event, and assumes the stream is sorted ascending by time.

Sparse point set¶

points = stream.pset()
ecv.pset(stream).numpy()        # [N, 4] float32 array: columns (x, y, t, p)

No parameters. Unlike the frame representations above, pset returns an EventPointSet, not an EventFrame: an [N, 4] array with x/y normalized to [0, 1] by sensor size, t normalized to [0, 1] over the stream’s span, and p as +1/-1. Because it’s sparse rather than a dense [C, H, W] tensor, it isn’t a valid choice for open(repr=...) / reader.with_repr(...) (those require a dense frame per slice).

Connected-component labels¶

frame = stream.count()
labels = frame.connected_components(connectivity=8)

connectivity (int, default 8) — 4 or 8-connectivity for the flood fill.

Single channel, uint64. This is a post-processing step on an existing EventFrame (typically a binary or count image), not a representation generated directly from a stream: any non-zero pixel (across all channels) is foreground, and each maximal connected blob gets a distinct label 1..k, with background 0.

By name¶

A representation can be picked by string anywhere a name is accepted: flatten(), view(), open(..., repr=...), and reader.with_repr(...). The name is one of "polarity", "binary", "count", "voxel", "tsurf", "atsurf", "tencode", "mcts", or "flow".

flatten, view, and open(repr=...) render the representation with its default parameters — they don’t accept a representation’s own parameters (bins, tau_ms, window_ms, max_window_ms, window), so stream.flatten("voxel", bins=5) is a TypeError. The one exception is normalize, which is an argument of flatten/view themselves, so stream.flatten("count", normalize=True) does work (and each representation keeps its own normalize default when the argument is omitted — False for count, True for polarity). To override the other parameters by name, use reader.with_repr(name, **opts), which returns a new dataset reader:

# defaults, by name:
stream.flatten("voxel")                                  # == stream.voxel()  (bins=9)
reader = ecv.open("huge.hdf5", dt_ms=30, repr="mcts")    # mcts, default max_window_ms=30

# parameters by name — only with_repr forwards them:
reader = ecv.open("huge.hdf5", dt_ms=30).with_repr("voxel", bins=5)
array = reader[0]                                        # [5, H, W] voxel for frame 0
frame = reader.slice(7).flatten("mcts")                  # name form on a stream → EventFrame

"pset" and "labels" are not valid names here — pset is sparse (see above), and labels is generated from a frame, not a stream.