Package io.datareplication.consumer

Interface StreamingPage<PageHeader extends ToHttpHeaders,​EntityHeader extends ToHttpHeaders>

  • Type Parameters:
    PageHeader - the type of the page header; in practice this will be either * SnapshotPageHeader or * FeedPageHeader
    EntityHeader - the type of the entity headers; see Entity
    All Superinterfaces:
    java.util.concurrent.Flow.Publisher<StreamingPage.Chunk<EntityHeader>>
    public interface StreamingPage<PageHeader extends ToHttpHeaders,​EntityHeader extends ToHttpHeaders>
extends java.util.concurrent.Flow.Publisher<StreamingPage.Chunk<EntityHeader>>

    The StreamingPage interface represents a streaming feed or snapshot page. It allows consuming a page incrementally without fully downloading it and keeping it in memory. By default, individual entity bodies are also streamed in chunks. The size of each data chunk is implementation-defined and usually corresponds to the buffer size used by the HTTP client implementation.

    StreamingPage extends the Flow.Publisher interface providing a stream of StreamingPage.Chunk objects. This is the lowest level of the interface and allows consuming entity bodies in small chunks. Alternatively, the toCompleteEntities() and toCompletePage() methods will return less granular representations (a stream of complete entities and a completely downloaded page respectively). These representations require keeping more data in memory at once (full entities and full pages respectively) but they're often easier to work with, trading memory efficiency for ease of use.

    Because of the stateful nature of this interface, each page can only be consumed in one of these ways. For example, after calling toCompleteEntities(), subscribing to the stream of chunks is impossible and will likely throw an error.

    • Method Detail

      • header

        @NonNull
PageHeader header()
        Returns:
        the page's headers

      • boundary

        @NonNull
@NonNull java.lang.String boundary()
        Returns:
        the boundary string for the page's multipart representation

      • toCompleteEntities

        @NonNull
default java.util.concurrent.Flow.Publisher<@NonNull Entity<EntityHeader>> toCompleteEntities()

        Wrap this page in a stream of complete Entity objects. The returned Flow.Publisher will collect the full body for an entity and return it all at once. Naturally, this requires buffering the entire body of each entity in memory, which might cause memory pressure for large entities.

        This method consumes the stream. After calling this method, subscribing to the page itself or calling either toCompletePage() or this method again is not allowed and will return an error.

        Returns:
        a Flow.Publisher providing a stream of complete Entity objects

      • toCompletePage

        @NonNull
default @NonNull java.util.concurrent.CompletionStage<@NonNull Page<PageHeader,​EntityHeader>> toCompletePage()

        Fully download this page and return a Page of its entities. The returned object contains the entire contents of the page in memory. Naturally, keeping all bodies of all entities on the page in memory might cause memory pressure for large pages.

        This method consumes the stream. After calling this method, subscribing to the page itself or calling either toCompleteEntities() or this method again is not allowed and will return an error.

        Returns:
        a Page containing the complete contents of this page