Arrow IPC

Files ending in .arrow, .ipc, or .feather are read with arrow. Write Arrow IPC output with -O arrow or by naming the output accordingly.

Reading

rush run -o flights.arrow 'df' flights.parquet
rush run 'head(df, 3) |> dplyr::select(carrier, origin, dest)' flights.arrow
#> carrier,origin,dest
#> UA,EWR,LAX
#> AA,JFK,SFO
#> DL,LGA,ATL

Writing

rush run -o subset.arrow 'dplyr::filter(df, dep_delay > 10)' flights.parquet
rush run 'nrow(df)' subset.arrow
#> 4

Using -O arrow with output flag

rush run -O arrow -o sample.arrow 'head(df, 5)' penguins.csv
rush run 'ncol(df)' sample.arrow
#> 8

Alternative extensions

The .feather and .ipc extensions are also recognized:

cp flights.arrow flights.feather
rush run 'nrow(df)' flights.feather
cp flights.arrow flights.ipc
rush run 'nrow(df)' flights.ipc
rm -f flights.feather flights.ipc
#> 10
#> 10