Cart
|
[XML-DEV Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message] Re: Converting a variety of data formats, containing variouski
 On 10/4/2016 11:53 AM, Costello, Roger L. wrote: One of the problems in tackling the question is that we don't know why all this data is supposed to end up in a common data store. The data are all about different kinds of things, so presumably there won't be an overlap between tables (just as an example, assuming the final data store even has tables). So why? Is it so that a single query language can be used over all the data? Is it so that there is only one kind of data system software to administer or maintain? Is it to satisfy some manager who insists on this scheme? Is it a research project to see if RDF is suitable for some task?Scenario: You are tasked to build a system that will receive data from a variety of sources. The data arrives in different formats (e.g., CSV, binary, tab-delimited, JSON, XML). The data sources provide different kinds of data (e.g., one source provides book data, another source provides weather data, another source provides gardening data). The data will be converted to a common intermediary form and then from the intermediary form, placed into a data store. But there are some things I notice that relate. Some of the example formats are essentially the same thing. e.g., CSV and tab-separated data really the same, and they both provide a row-oriented view of the data. They can be viewed as representing the contents of a relational table - each row is a relation. I'm assuming that each row contains the same columns throughout the file. We can't be sure a priory which column(s) represent a primary key, or if there even is one, and we can't be sure about foreign keys, either. There's your data conversion problem. As relations in a relational table, they can be denormalized into groups of triples, so basically RDF. XML, JSON, and binary can potentially contain nested data structures. So they have some real similarities, even if there could be specialized things that are common to one format, such as processing instructions in XML. That specialized data, and the knowledge about it, have to be put into the specific data converter. RDF can represent nested data structures. Again, the converter has to be designed with specialized knowledge to make this work. Roger quoted this claim: "RDF is the only data model the [sic] enables such conversions to be done in a way that is computationally tractable." I think that is silly on its face. However, if not RDF, then you really want some other existing standard, preferably one that has had a lot of development, and has practical software available including a query language. So why not use RDF, which does have all those things? Well, that might depend a lot on whether you need features that RDF is weak on, such as reification so that you can make metastatements about data provenance and history, for example. Since we don't know anything about the intended usage of this universal data store, can't go further to answer the question. But in view of the standards issue, and depending on the usage pattern, it does seem that RDF *could* be in the running. TomP 
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] |
PURCHASE STYLUS STUDIO ONLINE TODAY!Purchasing Stylus Studio from our online shop is Easy, Secure and Value Priced! 
Download The World's Best XML IDE!Accelerate XML development with our award-winning XML IDE - Download a free trial today! Subscribe in XML format
|
|||||||||
