[XML-DEV Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message]

Re: Search strategy that returns hits and context?

• From: "Liam R. E. Quin" <liam@fromoldbooks.org>
• To: "ml-dev@lists.xml.org" <xml-dev@lists.xml.org>
• Date: Thu, 10 Nov 2022 18:13:08 -0500

On Thu, 2022-11-10 at 14:48 +0000, Roger L Costello wrote:
> 
> I want to know if the document contains the string "abc".

What makes sense here depends to some extent on the size of the
document, and also on whether the search is to be case sensitive
(should a search for "boy" match "Boy" in the document, and the other
way round too?) and whether substrings are counted (does the document
text "amboyna" match the search "boy"?) and whether stemming and
accents are counted (what if we search for bøy and the document has
"böys"?)

Generally i prefer to use XQuery if i'm extracting only part of the
document, as the XQuery database might have a full-text index, and even
with only a few tens of gigabytes of text in the document will go much
faster than software that searches the entire input each time. On the
other hand if you only ever need to do this search once and then you
discard the document, the XQuery database would need to parse it in
order to load it, so you don't gain anything.

>  I want software to search the document for all occurrences of "abc"
> and I want the results of the search to show both the value that
> "abc" is part of and the value's context. In the above XML document
> there are two occurrences of "abc", one in the <ItemA> element and
> the other in the <ItemD> element. 
> 
> I can think of two ways to display the search results. I describe the
> two ways below. Are there other ways?
> 
> One way is to return, for each occurrence, the element value and the
> path to the element, e.g.,
> 
> abc - yes!
> /Test/Context1/ItemA
> 
> abc ... oky-dokey
> /Test/Context2/Context3/ItemD
> 
> Another way is to show the entire document, omitting the portions
> that do not contain the desired search value, e.g.,
> 
> <Test>
>     <Context1>
>         <ItemA>abc - yes!</ItemA>
>     </Context1>
>     <Context2>
>         <Context3>
>             <ItemD>abc ... oky-dokey</ItemD>
>         </Context3>
>     </Context2>
> </Test>

Another way is a keyword-in-context index - see e.g.
https://words.fromoldbooks.org/Search/?kw=boy

and another is to show the contaning paragraph, if that makes sense in
this application - click on one of the matches in the results for the
above search and the result expands to show the whole paragraph.

I used XQuery (baseX) and the ftcontains operator to do this.

liam

> 

-- 
Liam Quin, https://www.delightfulcomputing.com/
Available for XML/Document/Information Architecture/XSLT/
XSL/XQuery/Web/Text Processing/A11Y training, work & consulting.
Barefoot Web-slave, antique illustrations:  http://www.fromoldbooks.org

• References:
  • Search strategy that returns hits and context?
    • From: Roger L Costello <costello@mitre.org>