I/O Connectors
Apache Beam I/O connectors provide read and write transforms for the most popular data storage systems so that Beam users can benefit from native optimised connectivity. With the available I/Os, Apache Beam pipelines can read and write data from and to an external storage type in a unified and distributed way.
I/O connectors denoted via X-language have been made available using the Apache Beam multi-language pipelines framework.
I/O connectors denoted as Supported via Managed API can be accessed using the simplified managed I/O APIs for Java and Python.
Built-in I/O Connectors
This table provides a consolidated, at-a-glance overview of the available built-in I/O connectors.
| Connector Name | Source Supported | Sink Supported | Java | Python | Go | Typescript | Batch Supported | Streaming Supported | Supported via Managed API |
|---|---|---|---|---|---|---|---|---|---|
| FileIO | ✔ | ✔ | ✔ native | ✔ native | ✔ native | Not available | ✔ | ✔ | ✘ |
| AvroIO | ✔ | ✔ | ✔ native | ✔ native | ✔ native | ✔ via X-language | ✔ | ✔ | ✘ |
| TextIO (metrics) | ✔ | ✔ | ✔ native | ✔ native | ✔ native | ✔ via X-language | ✔ | ✔ | ✘ |
| TFRecordIO | ✔ | ✔ | ✔ native | ✔ native | Not available | Not available | ✔ | ✘ | ✘ |
| XmlIO | ✔ | ✔ | ✔ native | Not available | Not available | Not available | ✔ | ✘ | ✘ |
| TikaIO | ✔ | ✘ | ✔ native | Not available | Not available | Not available | ✔ | ✔ | ✘ |
| ParquetIO (guide) | ✔ | ✔ | ✔ native | ✔ native | ✔ native | ✔ via X-language | ✔ | ✘ | ✘ |
| ThriftIO | ✔ | ✔ | ✔ native | Not available | Not available | Not available | ✔ | ✘ | ✘ |
| HadoopFileSystem | ✔ | ✔ | ✔ native | ✔ native | Not available | ✔via X-language | ✔ | ✘ | ✘ |
| GcsFileSystem (metrics) | ✔ | ✔ | ✔ native | ✔ native | ✔ native | ✔via X-language | ✔ | ✘ | ✘ |
| LocalFileSystem | ✔ | ✔ | ✔ native | ✔ native | ✔ native | ✔via X-language | ✔ | ✘ | ✘ |
| S3FileSystem | ✔ | ✔ | ✔ native | ✔ native | Not available | ✔via X-language | ✔ | ✘ | ✘ |
| In-memory | ✘ | ✔ | ✘ | ✘ | ✔ native | ✘ | ✔ | ✘ | ✘ |
| KinesisIO | ✔ | ✔ | ✔ native | ✔ via X-language | Not available | Not available | ✔ | ✔ | ✘ |
| AmqpIO | ✔ | ✔ | ✔ native | Not available | Not available | Not available | ✔ | ✔ | ✘ |
| KafkaIO | ✔ | ✔ | ✔ native | ✔ via X-language | ✔ via X-language | ✔ via X-language | ✔ | ✔ | ✔ |
| PubSubIO | ✔ | ✔ | ✔ native | ✔ native | ✔ native | ✔ via X-language | ✔ | ✔ | ✘ |
| JmsIO | ✔ | ✔ | ✔ native | Not available | Not available | Not available | ✔ | ✔ | ✘ |
| MqttIO | ✔ | ✔ | ✔ native | Not available | Not available | Not available | ✔ | ✔ | ✘ |
| RabbitMqIO | ✔ | ✔ | ✔ native | Not available | Not available | Not available | ✔ | ✔ | ✘ |
| SqsIO | ✔ | ✔ | ✔ native | Not available | Not available | Not available | ✔ | ✔ | ✘ |
| SnsIO | ✘ | ✔ | ✔ native | Not available | Not available | Not available | ✔ | ✘ | ✘ |
| CassandraIO | ✔ | ✔ | ✔ native | Not available | Not available | Not available | ✔ | ✘ | ✘ |
| HadoopFormatIO (guide) | ✔ | ✔ | ✔ native | Not available | Not available | Not available | ✔ | ✔ | ✘ |
| HBaseIO | ✔ | ✔ | ✔ native | Not available | Not available | Not available | ✔ | ✘ | ✘ |
| HCatalogIO (guide) | ✔ | ✔ | ✔ native | Not available | Not available | Not available | ✔ | ✔ | ✘ |
| KuduIO | ✔ | ✔ | ✔ native | Not available | Not available | Not available | ✔ | ✘ | ✘ |
| SolrIO | ✔ | ✔ | ✔ native | Not available | Not available | Not available | ✔ | ✔ | ✘ |
| ElasticsearchIO | ✔ | ✔ | ✔ native | Not available | Not available | Not available | ✔ | ✔ | ✘ |
| BigQueryIO (guide) (metrics) | ✔ | ✔ | ✔ native | ✔ native | ✔
native ✔ via X-language | ✔ via X-language | ✔ | ✔ | ✔ |
| BigTableIO (metrics) | ✔ | ✔ | ✔ native | ✔
native (sink) ✔ via X-language | ✔
native (sink) ✔ via X-language | Not available | ✔ | ✔ | ✘ |
| DatastoreIO | ✔ | ✔ | ✔ native | ✔ native | ✔ native | Not available | ✔ | ✔ | ✘ |
| SnowflakeIO (guide) | ✔ | ✔ | ✔ native | ✔ via X-language | Not available | Not available | ✔ | ✘ | ✘ |
| SpannerIO | ✔ | ✔ | ✔ native | ✔ via X-language | ✔ native | Not available | ✔ | ✔ | ✘ |
| JdbcIO | ✔ | ✔ | ✔ native | ✔ via X-language | ✔ via X-language | Not available | ✔ | ✘ | ✘ |
| DebeziumIO | ✔ | ✘ | ✔ native | ✔ via X-language | ✔ via X-language | Not available | ✔ | ✔ | ✘ |
| MongoDbIO | ✔ | ✔ | ✔ native | ✔ native | ✔ native | Not available | ✔ | ✘ | ✘ |
| MongoDbGridFSIO | ✔ | ✔ | ✔ native | Not available | Not available | Not available | ✔ | ✘ | ✘ |
| RedisIO | ✔ | ✔ | ✔ native | Not available | Not available | Not available | ✔ | ✘ | ✘ |
| DynamoDBIO | ✔ | ✔ | ✔ native | Not available | Not available | Not available | ✔ | ✔ | ✘ |
| ClickHouseIO | ✘ | ✔ | ✔ native | Not available | Not available | Not available | ✔ | ✘ | ✘ |
| DatabaseIO | ✔ | ✔ | ✘ | ✘ | ✔ native | Not available | ✔ | ✘ | ✘ |
| GenerateSequence | ✔ | ✘ | ✔ native | Not available | Not available | Not available | ✔ | ✔ | ✘ |
| SplunkIO | ✘ | ✔ | ✔ native | Not available | Not available | Not available | ✔ | ✔ | ✘ |
| FhirIO | ✔ | ✔ | ✔ native | Not available | ✔ native | Not available | ✔ | ✔ | ✘ |
| HL7v2IO | ✔ | ✔ | ✔ native | Not available | Not available | Not available | ✔ | ✔ | ✘ |
| DicomIO | ✔ | ✔ | ✔ native | ✔ native | Not available | Not available | ✔ | ✔ | ✘ |
| FlinkStreaming ImpulseSource | ✔ | ✘ | Not available | ✔ native | Not available | Not available | ✔ | ✔ | ✘ |
| Firestore IO | ✔ | ✔ | ✔ native | Not available | Not available | Not available | ✔ | ✘ | ✘ |
| Neo4j | ✔ | ✔ | ✔ native | Not available | Not available | Not available | ✔ | ✘ | ✘ |
| Pub/Sub Lite | ✔ | ✔ | ✔ native | ✔ via X-language | Not available | ✔ via X-language | ✔ | ✔ | ✘ |
| InfluxDB | ✔ | ✔ | ✔ native | Not available | Not available | Not available | ✔ | ✔ | ✘ |
| SparkReceiverIO (guide) | ✔ | ✘ | ✔ native | Not available | Not available | Not available | ✘ | ✔ | ✘ |
| CdapIO (guide) | ✔ | ✔ | ✔ native | Not available | Not available | Not available | ✔ | ✔ | ✘ |
| SingleStoreDB (guide) | ✔ | ✔ | ✔ native | Not available | Not available | Not available | ✔ | ✘ | ✘ |
| GoogleAdsIO | ✔ | ✔ | ✔ native | Not available | Not available | Not available | ✔ | ✔ | ✘ |
| Web APIs (guide) | ✔ | ✔ | ✔ native | ✔ native | Not available | Not available | ✔ | ✔ | ✘ |
| Iceberg (Managed I/O) | ✔ | ✔ | ✔ native | ✔ via X-language | Not available | Not available | ✔ | ✔ | ✔ |
Other I/O Connectors for Apache Beam
| Connector Name | Source Supported | Sink Supported | Java | Python | Go | Typescript | Batch Supported | Streaming Supported |
|---|---|---|---|---|---|---|---|---|
| Solace | ✔ | ✘ | ✔ native | Not available | Not available | Not available | ✔ | ✔ |
| SAP Hana to Google BigQuery | ✔ | ✔ | ✔ native | Not available | Not available | Not available | ✔ | ✘ |
| MySQL | ✔ | ✔ | Not available | ✔ native | Not available | Not available | ✔ | ✘ |
| TrepWsIO | ✔ | ✘ | ✔ native | Not available | Not available | Not available | ✔ | ✔ |
| KineticaDB | ✔ | ✔ | ✔ native | Not available | Not available | Not available | ✔ | ✘ |
| Cognite Data Fusion | ✔ | ✔ | ✔ native | Not available | Not available | Not available | ✔ | ✔ |
| Pyodbc | ✔ | ✔ | Not available | ✔ native | Not available | Not available | ✔ | ✘ |
| Go Connect | ✔ | ✔ | ✘ | ✘ | ✔ native | Not available | ✔ | ✔ |
| Tinybird | ✘ | ✔ | Not available | ✔ native | Not available | Not available | ✔ | ✔ |
| Cloud SQL | ✔ | ✘ | Not available | ✔ native | Not available | Not available | ✔ | ✘ |
| Cloud Bigtable (HBase based) | ✔ | ✔ | ✔ native | Not available | Not available | Not available | ✔ | ✘ |
| Beam PyIO (Collection of Python IO connectors) | ✔ | ✔ | Not available | ✔ native | Not available | Not available | ✔ | ✔ |
Last updated on 2026/02/09
Have you found everything you were looking for?
Was it all useful and clear? Is there anything that you would like to change? Let us know!