Skip to content

External Integrations

Ossify integrates with external data sources and analysis platforms to streamline neuromorphological data workflows.

Overview

Integration Category Functions Purpose
CAVE Integration load_cell_from_client, load_cell_batch_from_client, fetch_frames_batch Connectome analysis via CAVE infrastructure

CAVE Integration

CAVE (Connectome Analysis Versioning Engine) provides an interface for managing and analysis in large-scale densely segmented anatomical datasets datasets. Ossify integrates with CAVE to import neurons with their meshes, skeletons, and synaptic connectivity. If you have previously used pcg_skel, this is the equivalent of get_meshwork_from_client to import a neuron and synapses from the CAVE skeleton service.

load_cell_from_client

load_cell_from_client 

load_cell_from_client(root_id: int, client: CAVEclientFull, *, synapses: bool = False, reference_tables: Optional[list[str]] = None, reference_suffixes: Optional[dict] = None, restore_graph: bool = False, restore_properties: bool = True, synapse_spatial_point: str = 'ctr_pt_position', include_partner_root_id: bool = False, timestamp: Optional[datetime] = None, omit_self_synapses: bool = True, skeleton_version: int = 4, pre_syn_df: Optional[DataFrame] = None, post_syn_df: Optional[DataFrame] = None, l2_df: Optional[DataFrame] = None, skeleton: Optional[dict] = None, assume_valid: bool = False) -> Cell

Import an "L2" skeleton and spatial graph using the CAVE skeleton service.

Parameters:

  • root_id (int) –

    The root ID of the cell to import.

  • client (CAVEclientFull) –

    The CAVE client to use for data retrieval.

  • synapses (bool, default: False ) –

    Whether to include synapse information in the imported cell. Default is False.

  • reference_tables (Optional[list[str]], default: None ) –

    A list of table names to include as reference tables for synapse annotation. These will be merged into the synapse DataFrame if synapses=True.

  • restore_graph (bool, default: False ) –

    Whether to restore the complete spatial graph for the imported cell. Default is False. Setting to True will include all graph edges, but can take longer to process.

  • restore_properties (bool, default: True ) –

    Whether to restore all graph vertex properties of the imported cell. Default is False.

  • synapse_spatial_point (str, default: 'ctr_pt_position' ) –

    The spatial point column name for synapses. Default is "ctr_pt_position".

  • include_partner_root_id (bool, default: False ) –

    Whether to include the synaptic partner root ID from the imported cell. Default is False. If including partner root id, you are encouraged to set a timestamp to ensure consistent results. Otherwise, querying different cells at different points in time can result in different results for partner root ids.

  • timestamp (Optional[datetime], default: None ) –

    The timestamp to use for the query. If not provided, the latest timestamp the root id is valid will be used.

  • omit_self_synapses (bool, default: True ) –

    Whether to omit self-synapses from the imported cell. Default is True, since most are false detections.

  • skeleton_version (int, default: 4 ) –

    The skeleton service data version to use for the query. Default is 4.

  • pre_syn_df (Optional[DataFrame], default: None ) –

    Pre-fetched pre-synapse frame. When provided (and synapses=True), the internal pre-synapse query is skipped and this frame is used as-is. Must be in the exact shape the internal fetch produces: autapses omitted, drop_other_side/ include_partner_root_id applied, the pre_pt_l2_id column present, and any reference columns already merged. Used by :func:fetch_frames_batch.

  • post_syn_df (Optional[DataFrame], default: None ) –

    Pre-fetched post-synapse frame; same contract as pre_syn_df (post_pt_l2_id).

  • l2_df (Optional[DataFrame], default: None ) –

    Pre-fetched L2 property frame. When provided, the internal get_l2data_table call is skipped. Must already be reset_index()'d (l2_id as a column), row-ordered to match sk["lvl2_ids"], and contain the attribute set implied by restore_properties.

  • skeleton (Optional[dict], default: None ) –

    Pre-fetched skeleton dict (as returned by client.skeleton.get_skeleton(..., output_format="dict")). When provided, the internal get_skeleton call is skipped. Used by :func:load_cell_batch_from_client to avoid re-fetching a skeleton already pulled to obtain lvl2_ids.

  • assume_valid (bool, default: False ) –

    When True, skip the timestamp/validity round trip and use timestamp directly (which must be provided). Set by :func:load_cell_batch_from_client after validating the whole batch once. With skeleton, l2_df and the synapse frames all injected, this makes assembly fully network-free.

Returns:

  • Cell –

    The imported cell object.

Import neurons directly from CAVE databases with automatic skeleton generation, graph reconstruction, and synapse mapping.

Prerequisites

# Install CAVE client
pip install caveclient
``

#### Basic Usage

```python
import ossify
from caveclient import CAVEclient

# Initialize CAVE client
client = CAVEclient("minnie65_public")  # MICrONS dataset

# Import neuron with basic skeleton
cell = ossify.load_cell_from_client(
    root_id=864691135336055529,
    client=client
)

print(f"Loaded cell {cell.name}")
print(f"Skeleton vertices: {cell.skeleton.n_vertices}")
print(f"Graph vertices: {cell.graph.n_vertices}")

Advanced Import Options

from datetime import datetime

# Import with synapses and full graph
cell = ossify.load_cell_from_client(
    root_id=864691135336055529,
    client=client,
    synapses=True,                    # Include synapse annotations
    restore_graph=True,               # Include complete L2 graph
    restore_properties=True,          # Include all vertex properties
    include_partner_root_id=True,     # Include synaptic partner IDs
    omit_self_synapses=True,          # Remove autapses
    skeleton_version=4                # Skeleton service version
)

# Check imported data
print(f"Graph vertices: {cell.graph.n_vertices}")
print(f"Skeleton vertices: {cell.skeleton.n_vertices}")
print(f"Presynaptic sites: {len(cell.annotations['pre_syn'])}")
print(f"Postsynaptic sites: {len(cell.annotations['post_syn'])}")
print(f"Available features: {cell.skeleton.features.columns.tolist()}")

Synapse Analysis Workflow

# Import cell with synapses
cell = ossify.load_cell_from_client(
    root_id=864691135336055529,
    client=client,
    synapses=True,
    timestamp=datetime(2023, 6, 1)  # Consistent analysis timestamp
)

# Analyze compartmentalization
is_axon, segregation = ossify.label_axon_from_synapse_flow(
    cell, 
    return_segregation_index=True
)

compartment = ["dendrite" if not ax else "axon" for ax in is_axon]
cell.skeleton.add_feature(compartment, "compartment")

print(f"Segregation index: {segregation:.3f}")
print(f"Axon fraction: {is_axon.mean():.2%}")

# Visualize results
fig, ax = ossify.plot_cell_2d(
    cell,
    color="compartment",
    palette={"axon": "red", "dendrite": "blue"},
    synapses=True,
    pre_color="orange",
    post_color="green",
    units_per_inch=100_000 # nm
)

load_cell_batch_from_client

load_cell_batch_from_client 

load_cell_batch_from_client(root_ids: list[int], client: CAVEclientFull, *, synapses: bool = False, reference_tables: Optional[list[str]] = None, reference_suffixes: Optional[dict] = None, restore_graph: bool = False, restore_properties: bool = True, synapse_spatial_point: str = 'ctr_pt_position', include_partner_root_id: bool = False, timestamp: Optional[datetime] = None, omit_self_synapses: bool = True, skeleton_version: int = 4, skip_invalid: bool = False, skeleton_download_method: Literal['gcs', 'server'] = 'gcs', row_limit: int = 500000) -> dict[int, Cell]

Load many cells with the poolable fetches batched into a few queries.

Equivalent to calling :func:load_cell_from_client on each root id, but the synapse and L2-property fetches are pooled across the whole batch (see :func:fetch_frames_batch) and each cell is then assembled network-free — no skeleton is fetched twice, and the batch is validated with a single round trip. All roots share one timestamp.

All parameters not listed below match :func:load_cell_from_client and are applied identically to every cell in the batch.

Parameters:

  • root_ids (list[int]) –

    Root ids to load.

  • timestamp (Optional[datetime], default: None ) –

    Single shared timestamp for the batch. If None, the current materialization timestamp (client.materialize.get_timestamp()) is used — pin client.version for a reproducible batch.

  • skip_invalid (bool, default: False ) –

    If True, roots that are not valid at timestamp — or whose skeleton is not available — are dropped from the result instead of raising.

  • skeleton_download_method (Literal['gcs', 'server'], default: 'gcs' ) –

    How fetch_skeletons retrieves cached skeletons. "gcs" (default) downloads H5 files directly from the storage bucket via a downscoped token, bypassing the service for data transfer and avoiding its request rate limits — preferred for bulk loads. "server" routes the download through the skeleton service instead.

  • row_limit (int, default: 500000 ) –

    Passed to :func:fetch_frames_batch; guards against a silently-truncated pooled synapse query at/above the server row limit (default 500,000). Set to 0 to disable.

Returns:

  • dict[int, Cell] –

    Mapping of root id to the assembled :class:~ossify.base.Cell. Invalid roots are absent when skip_invalid=True.

Load many cells at once, pooling the synapse and L2 queries across the whole batch and downloading skeletons in a single bulk request. Each returned Cell is identical to what load_cell_from_client produces for that root id, but a batch of N cells costs a handful of round trips instead of ~9 × N.

root_ids = [864691135639806264, 864691135639806265, 864691135639806266]

cells = ossify.load_cell_batch_from_client(
    root_ids,
    client,
    synapses=True,
    reference_tables=["synapse_target_predictions_ssa_v2"],
    reference_suffixes={"synapse_target_predictions_ssa_v2": "ssa"},
    timestamp=timestamp,     # one shared timestamp for the batch
    skip_invalid=True,       # drop invalid/unskeletonized roots instead of raising
)

for root_id, cell in cells.items():
    print(root_id, cell.skeleton.n_vertices)

Recommended batch size ~10

Around 10 cells per batch is a good default: past that the pooled queries are payload-bound (per-cell time stops improving), and ~10 stays under the server's 500,000-row query limit even for heavily-connected cells while matching the synchronous bulk-skeleton download cap. For large jobs, generate skeletons up front with client.skeleton.generate_bulk_skeletons_async (up to 10,000 ids/call), then load small batches against the warm cache. See the Data Import and Export guide for the full workflow.

fetch_frames_batch

fetch_frames_batch 

fetch_frames_batch(root_ids: list[int], client: CAVEclientFull, *, synapses: bool = True, reference_tables: Optional[list[str]] = None, reference_suffixes: Optional[dict] = None, include_partner_root_id: bool = False, omit_self_synapses: bool = True, restore_properties: bool = True, timestamp: Optional[datetime] = None, lvl2_ids_by_root: Optional[dict] = None, row_limit: int = 500000) -> dict[int, dict]

Batch-fetch the poolable per-cell frames for many root ids in a few queries.

Collapses the two dominant per-cell fetches of :func:load_cell_from_client — synapse queries and L2 property lookups — into a handful of pooled requests, then slices the results back per root id. The returned frames are byte-identical to what the single-cell path produces and can be handed straight to load_cell_from_client via its pre_syn_df / post_syn_df / l2_df injection params.

All roots share a single timestamp (pass an explicit value or pin client.version); the pooled filter_in queries require one consistent materialization for the batch.

Parameters:

  • root_ids (list[int]) –

    Root ids to fetch. Order is preserved in the scatter-back.

  • client (CAVEclientFull) –

    The CAVE client to use.

  • synapses (bool, default: True ) –

    Whether to fetch synapse frames. Default True. When False, only l2_df is populated (requires lvl2_ids_by_root).

  • reference_tables (Optional[list[str]], default: None ) –

    Same meaning as in :func:load_cell_from_client; applied identically per root.

  • reference_suffixes (Optional[list[str]], default: None ) –

    Same meaning as in :func:load_cell_from_client; applied identically per root.

  • include_partner_root_id (Optional[list[str]], default: None ) –

    Same meaning as in :func:load_cell_from_client; applied identically per root.

  • omit_self_synapses (Optional[list[str]], default: None ) –

    Same meaning as in :func:load_cell_from_client; applied identically per root.

  • restore_properties (bool, default: True ) –

    When True, fetch all L2 attributes; otherwise only rep_coord_nm.

  • timestamp (Optional[datetime], default: None ) –

    The single shared timestamp for the batch.

  • lvl2_ids_by_root (Optional[dict], default: None ) –

    Mapping {root_id: sk["lvl2_ids"]} from the caller's skeleton dicts. Required to fetch L2 data; the pooled get_l2data_table runs over the union and each root's frame is sliced back in lvl2_ids order (dropping cache-missing ids), matching the single-cell path exactly. If omitted, l2_df is left None for every root.

Returns:

  • dict[int, dict] –

    {root_id: {"pre_syn_df": ..., "post_syn_df": ..., "l2_df": ...}}. Entries that were not fetched (e.g. synapses off) are None.

Lower-level helper used by load_cell_batch_from_client. It returns the pooled per-root synapse and L2 frames ({root_id: {"pre_syn_df", "post_syn_df", "l2_df"}}) without assembling Cell objects, which are byte-identical to the single-cell path. Most users should call load_cell_batch_from_client; reach for this only when you want the raw frames — for example to feed them into load_cell_from_client via its pre_syn_df / post_syn_df / l2_df injection parameters in a custom pipeline.

Cross-Platform Analysis Pipeline

def cave_to_analysis_pipeline(root_ids, client, output_format="both"):
    \"\"\"Complete pipeline from CAVE import to analysis results\"\"\"

    results = {}

    for root_id in root_ids:
        print(f"Processing cell {root_id}...")

        # Import from CAVE
        cell = ossify.load_cell_from_client(
            root_id=root_id,
            client=client, 
            synapses=True,
            restore_properties=True
        )

        # Morphological analysis
        strahler = ossify.strahler_number(cell)
        cell.skeleton.add_feature(strahler, "strahler_order")

        # Compartment analysis
        is_axon, segregation = ossify.label_axon_from_synapse_flow(
            cell, return_segregation_index=True
        )
        compartment = ["dendrite" if not ax else "axon" for ax in is_axon]
        cell.skeleton.add_feature(compartment, "compartment")

        # Generate visualization
        fig, axes = ossify.plot_cell_multiview(
            cell,
            color="compartment",
            palette={"axon": "red", "dendrite": "blue"},
            synapses=True,
            units_per_inch=100000
        )

        # Save results
        if output_format in ["ossify", "both"]:
            ossify.save_cell(cell, f"cell_{root_id}.osy")

        if output_format in ["figure", "both"]:
            fig.savefig(f"cell_{root_id}_analysis.pdf", dpi=300, bbox_inches='tight')

        # Store metrics
        results[root_id] = {
            'total_length_um': cell.skeleton.cable_length() / 1000,
            'n_synapses': len(cell.annotations["pre_syn"]) + len(cell.annotations["post_syn"]),
            'segregation_index': segregation,
            'axon_fraction': is_axon.mean()
        }

        print(f"  Length: {results[root_id]['total_length_um']:.1f} μm")
        print(f"  Synapses: {results[root_id]['n_synapses']}")
        print(f"  Segregation: {segregation:.3f}")

    return results

# Run complete pipeline
root_ids = [864691135336055529, 864691135174324866]
analysis_results = cave_to_analysis_pipeline(root_ids, client, "both")

CAVE Integration Features

Multi-Scale Data: Automatically combines L2 graph connectivity with skeleton representations.

Temporal Consistency: Support for timestamp-locked analyses across datasets.

Synapse Mapping: Automatic mapping of synaptic sites to skeleton structures.

Quality Control: Built-in validation for data integrity and biological plausibility via compartments.

Best Practices for CAVE Integration

  • Use Timestamps: Always specify timestamps for reproducible analyses
  • Batch Processing: Use load_cell_batch_from_client (batches of ~10) to load many cells with pooled queries instead of looping load_cell_from_client
  • Error Handling: Implement robust error handling for network operations
  • Data Validation: Validate imported data before analysis
  • Version Control: Track skeleton service and dataset versions used
  • Memory Management: Consider memory usage when importing large datasets