XML Editor - Download a Free Trial >
See What's New >
Buy Now >

[XML-DEV Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message]

Re: What is the general direction you are seeing these daysto

  • From: Ihe Onwuka <ihe.onwuka@gmail.com>
  • To: "Liam R. E. Quin" <liam@w3.org>
  • Date: Sun, 8 Mar 2015 17:36:57 -0400
Re: What is the general direction you are seeing these daysto
Play the video


On Sun, Mar 8, 2015 at 3:00 PM, Ihe Onwuka <ihe.onwuka@gmail.com> wrote:

On Sat, Mar 7, 2015 at 9:05 PM, Liam R. E. Quin <liam@w...> wrote:

Sometimes it's about extracting just the bits you need to query and
putting just those in some kind of database - relational, NoSQL,
SlightlySQL, triple store, XML-native, hybrid, whatever, with no hope
of reconstructing the original -- if you need the original, just use
it directly.

Depends. If you are constructing a repository from a denormalized text dump and/or disparate sources you may not want or be able to reconstruct the original.

Actually I can concretize that. It describes most of the databases behind most public movie API's, e.g http://omdbapi.com/. Typically such databases consist of pieces of information cobbled together from various places usually they run on a LAMPish/Java platform. There are research papers detailing the challenges encountered in  such an endeavour 


Extraction and integration of MovieLens and ImDb data

I would expect similar  problems (matching to support correct aggregation, deduplication, disambiguation) to manifest in many Big Data projects simply because  data integration has the potential to add so much value. A "Small Data" project may morph into a Big Data one once you have assembled  information from enough sources/silos. 

I've been allowed to approach this problem with NXD technology. Schemalessness - being able to ingest data from anywhere in any format - and the ability to defer decisions on data management due to a superior extract and transform capability, has been key to dodging many of the problems that the MovieLens group experienced. It has also allowed the focus to shift to more ambitious higher value problems, although challenges remain.






 
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index]

PURCHASE STYLUS STUDIO ONLINE TODAY!

Purchasing Stylus Studio from our online shop is Easy, Secure and Value Priced!

Buy Stylus Studio Now

Download The World's Best XML IDE!

Accelerate XML development with our award-winning XML IDE - Download a free trial today!

Don't miss another message! Subscribe to this list today.
Email
First Name
Last Name
Company
Subscribe in XML format
RSS 2.0
Atom 0.3
 

Stylus Studio has published XML-DEV in RSS and ATOM formats, enabling users to easily subcribe to the list from their preferred news reader application.


Stylus Studio Sponsored Links are added links designed to provide related and additional information to the visitors of this website. they were not included by the author in the initial post. To view the content without the Sponsor Links please click here.

XML Editor - Download a 15 Day Free Trial Now >
See What's New in Stylus Studio >
Buy Stylus Studio - XML Editor - Now >
Site Map | Privacy Policy | Terms of Use | Trademarks
Free Stylus Studio XML Training:
W3C Member
Stylus Studio® and DataDirect XQuery ™are products from DataDirect Technologies, is a registered trademark of Progress Software Corporation, in the U.S. and other countries. © 2004-2013 All Rights Reserved.