Streaming In Mule

  • Allows flows to consume very large messages in an efficient way
  • Message payloads are not read into memory
  • Simple routing rules based on message metadata are still possible
  • You can combine streaming and non-streaming endpoints.
  • In the following example, the flow shows the HTTP Listener source that receives a POST method with a body payload to write to the files.
  • The flow writes the first file correctly, while the second file is created with empty content because each component that consumes a stream expects to receive a new stream.
  • After the first File Write operation consumes the stream, the second File Write operation receives an empty stream.
  • Thereby, the second operation has no content to write to a file.
  • Read a stream more than once.
  • Have concurrent access to the stream.
  • Repeatable file store stream (by default, this strategy is selected).
  • Non repeatable stream.
  • Repeatable in-memory stream.
  • DataWeave processes streamed data as its bytes arrive instead of scanning the entire document to index it.
  • When in deferred mode, DataWeave can also pass streamed output data directly to a message processor without saving it to the disk.
  • This behavior enables DataWeave and Mule to process data faster and consume fewer resources than the default processes for reading and writing data.
  • The basic unit of the stream is specific to the data format.
  • The unit is a record in a CSV document, an element of an array in a JSON document, or a collection in an XML document.

Streaming XML

XML is more complicated than JSON because there are no arrays in the document.

--

--

Love podcasts or audiobooks? Learn on the go with our new app.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store