# `File.stream!/1` processes large files with constant memory

`File.read!/1` loads the entire file into memory. A 2GB log file? That's 2GB of RAM. Use `File.stream!/1` to process files line-by-line with constant memory usage, regardless of file size.

```elixir
# Memory usage grows with file size
File.read!("huge_file.csv")                    # Loads entire file into memory
|> String.split("\n")                          # Doubles memory (original + split)
|> Enum.map(&parse_row/1)                      # Triples memory (adds parsed data)

# Constant memory regardless of file size
File.stream!("huge_file.csv")                  # Opens file handle, reads nothing
|> Stream.map(&String.trim/1)                  # Lazy - processes one line at a time
|> Stream.map(&parse_row/1)                    # Still lazy - no intermediate lists
|> Enum.to_list()                              # Only now does work happen
```

Streams are lazy. Nothing executes until you call an eager function like `Enum.to_list/1` or `Enum.each/1`. Each line flows through the entire pipeline before the next line is read.

## Processing in chunks for batch operations

```elixir
"events.jsonl"
|> File.stream!()
|> Stream.map(&Jason.decode!/1)
|> Stream.chunk_every(1000)
|> Enum.each(fn batch ->
  Repo.insert_all(Event, batch)
end)
```

`Stream.chunk_every/2` groups lines into batches—perfect for bulk database inserts or API calls with rate limits.

## Reading binary files

```elixir
# Read 64KB chunks instead of lines
File.stream!("video.mp4", [], 65_536)
|> Enum.reduce(0, fn chunk, acc -> acc + byte_size(chunk) end)
```

The third argument controls chunk size in bytes. Use this for binary files where line-based reading doesn't make sense.


---

Created by: almirsarajcic
Date: December 03, 2025
URL: https://elixirdrops.net/d/V6BducLZ