Re: How to stream-process non-XML text using unparsed

On Thu, Jul 24, 2014 at 10:07 AM,  <mike@xxxxxxxxxxxx> wrote:
> let $x := unparsed-text-lines(...)
>
> return ($x[1000], $x[10])
>
> Saxon implements $x using a data structure called a MemoClosure where items
> from the input are read on-demand, and then remembered. Reading $x[1000]
> will cause the first 1000 items to be read and retained in memory; reading
> $x[10] finds that the tenth item is already in memory



 Maybe not store the 1000 lines, except the last, but just create a
map N --> offset(N) for the Nth line, N < 1000

This way, if we store what was read (or if it already is conveniently
accessible by offset), then we will retrieve it again only on demand.
Reminds me of <xsl:key> and the key() function -- and maybe something
like key() for streamed data could be designed this way -- obviously
requiring two (partial/on-demand) passes.

Just thinking aloud about this.

-- 
Cheers,
Dimitre Novatchev
---------------------------------------
Truly great madness cannot be achieved without significant intelligence.
---------------------------------------
To invent, you need a good imagination and a pile of junk
-------------------------------------
Never fight an inanimate object
-------------------------------------
To avoid situations in which you might make mistakes may be the
biggest mistake of all
------------------------------------
Quality means doing it right when no one is looking.
-------------------------------------
You've achieved success in your field when you don't know whether what
you're doing is work or play
-------------------------------------
To achieve the impossible dream, try going to sleep.
-------------------------------------
Facts do not cease to exist because they are ignored.
-------------------------------------
Typing monkeys will write all Shakespeare's works in 200yrs.Will they
write all patents, too? :)
-------------------------------------
I finally figured out the only reason to be alive is to enjoy it.