Storage of XML data

Hi,

I've just finished reading an XML spec/schema for news stories. The concept seems to be very interesting, especially splitting style and content.

I am having problems working how to store XMLised data in a MySQL structure.

With embedded entities and a generally open structure it seems impossible to properly store this data in a flat-ish structure such as MySQL.

Where is XML meant to fit in ?

It is a transport layer .... take data from a database and convert it XML and then onto the client

Or is it for data storage / encapsulation as well ?

It's really testing the old grey matter, hopefully the penny will drop soon

Also has any1 any ideas on how to prepare for full XML useage while still maintaining compatibility with old HTML browsers.

Thanks for any comments..

I have read article after article - even 'tutorial' after 'tutorial' (quoted because they often do not actually teach you anything) on XML - and I am still buggered as to what I am supposed to do with it?

I have read all about the librarys of DTD's which authors always seem to forget the link to, well-formed vs ..... (I forget), XHTML? , XSL,,,....!!!!!

Apparantly it will help search engines to continue their fight to ignore all genuine sites.

<well-formed-cynic>Simon</well-formed-cynic>

------------------
Simon Wheeler
FirePages -DHTML/PHP/MySQL

Hi ya,

Every one I talk to is bitching about XML. Glad I'm not the only one having problems with it all.

I have daily arguments with the Tech Manager at work. He loves XML (although has done nothing with it) and swears it is the future. His take on it all is that it is a 'transport layer', you take your data out of database, convert it to XML, send it the client, convert it back to useable data, merge it with any style and formatting, display it.


XML is also very hungry on bandwidth. Sending all the data wrapped in humanised plain text is not a very efficient way of doing things. Even a blank wap-WML page seems to take at least 10k because of DTDs etc.

I'm off to get the full W3C docs and see what exactly they say about it. It seems like the 'steering committee' may have a taken a dead end...or at least a dirt track.

Any one got any <working>XML</working> sites out there ?

i am also glad to see that i am not the only one that doesnt see where the hype actaully comes into play. I have been doing some work with XML, but so far my main questions is: "what advantage here is so great that i should stop everything and start to do my site in it?" Oh wow you can make your own tags and send them through your XSL doc, thats cool but not much else as i see it at this point. Maybe i am missing something (in fact i hope that i am missing something). I keep hearing "XML is huge, it is going to change the way we do everything", yet no one really seems to know how or why its so huge.

Any insight?

Hi,

OK, heres a quick example ... I know the XML is wrong etc.. its just an example

A News Story

<news>
<date>21st June 2000</date>
<author>Me</author>
<story>
<date>Yesterday</date><person>Mr Joe Bloggs</person> had a massive <event>heart attack</event> after trying to get his head round XML.
</story>
</news>

This seems OK, but how do I now import that data into a MySQL database and then back out again as XML. I guess you will not store the tags in the database, and these will need to be inserted when it is sent back out.

Surely for every word / phrase it will have to search the database to try and find out what type of data that is and wrap it in a tag.

Am I going down the right road ????? or lost the point completely ... if there ever was a point in the first place.

Thanks..

Al depends on yer <forgot the acronym>DTD's</forgot the acronyn> I suppose,

At Marthasonlinedating.com the DTD specifies the <date> tag as a pleasant social encounter with a member of the opposite sex, of course the middle eastern fruit importers.com insist that <date> is of course a kind of <from a tree><fruit type="sticky"><date>pudding</date></fruit></from a tree>.

Tis possible that Marthasonline dating may misread<event>heart attack</event> incorrectly and send him a bunch of roses instead of an ambulance ??

OK its obvious I know little of XML but please do not tell us it will make our life any easier!.


Simon.

------------------
Simon Wheeler
FirePages -DHTML/PHP/MySQL

This place http://www.w3schools.com/xml/default.asp
seems to have some good information. I went through the wml/wap section last week and that seemed OK for as far as it went.
CML is the best example of XML in action. Like WML it's an industry agreed upon mark up language.

Robman

I forgot to mention SVG the new graphics format. It's XML based. Check out SVG at the W3C it looks pretty cool.

Robman

Also at Webmonkey:
http://hotwired.lycos.com/webmonkey....html?tw=design

Robman

You are completely on the RIGHT TRACK!

how does one get XML into mySQL. where the first alternative is to use an XML based database like eXcelon if you have $15K. if you don't then i see two options.

1) use the great XML modules (like XML::Parser) for Perl available at search.cpan.org. this will allow you to easily break up the data and store them in appropriate columns.

2) if searching is not a big deal then just store the XML as a glob and when you want to retrieve it, get the XML data and use XQL and XSL to query and present the data, respectively. XSL allows you to turn it into XML, HTML, WML, txt, whatever in a routine manner.


good luck.


<BLOCKQUOTE><font size="1" face="Verdana,Arial,Helvetica">quote:</font><HR>Originally posted by dynamic1:
Hi,

OK, heres a quick example ... I know the XML is wrong etc.. its just an example

A News Story

<news>
<date>21st June 2000</date>
<author>Me</author>
<story>
<date>Yesterday</date><person>Mr Joe Bloggs</person> had a massive <event>heart attack</event> after trying to get his head round XML.
</story>
</news>

This seems OK, but how do I now import that data into a MySQL database and then back out again as XML. I guess you will not store the tags in the database, and these will need to be inserted when it is sent back out.

Surely for every word / phrase it will have to search the database to try and find out what type of data that is and wrap it in a tag.

Am I going down the right road ????? or lost the point completely ... if there ever was a point in the first place.

Thanks..[/quote]




[This message has been edited by bluginger (edited June 19, 2000).]

------------------------
This seems OK, but how do I now import that data into a MySQL database and then back out again as XML. I guess you will not store the tags in the database, and these will need to be inserted when it is sent back out.

Surely for every word / phrase it will have to search the database to try and find out what type of data that is and wrap it in a tag.
---------------------------------------
I guess we are all losing the point here. First of all there is a big point. When the data in your site changes daily, then it makes a VERY good sense to write XML.

Since you define the data, so you make the rules in your parser program to parse the tagged XML data and read the data (not the tags). This is done thu a spec called XML DOM manipulation. Its same as any other DOM object manupulation. By doing the necessay commands, we can read the 'DOM tree' i.e the tagged nodes and accordingly make a insert into a database of the data as fields in to mysql database.

Its a short way to explain a large topic.
But XML shows is brilliance and ease pf site maintenance in a very data volatile environment where the data changes quite often but is of the same structure.

I've read a lot about XML, but I don't see the big deal. From what I can understand, it's not meant to be integrated with databases, rather it acts as its own database. The best use that I could find for it is to allow other websites to display your website's dynamic information in their format, such as news.


------------------
Aristotle concluded that those who admit their stupidity are truly wise.
-
Gil Hildebrand Jr
4atcost.com Senior Web Developer

For those who would like to see a good example of XML at work, try www.gingell.com

Craig has coded an xml version of his family geneology.

There is a time and a place for everything and although XML has got a lot going for it and is going to become more and more common place (just see how many new books have appeared in the last 12 months on the subject) that doesn't mean that one shoe fits all.

If your data will work very happily using a normal relational database representation then use that - don't bother with XML. Just so long as you can write queries that give you your data when, where and how you need it then that's fine URL.

Now there are other times where your data can be better or more flexibly described by XML than in a relational database. Or for transport purposes maybe an XML file is better than plain text, then again a plain old HTML file might just do the job.

I'll give you an example of how I am using both MySQL and XML together in my work. I sell stone for a living and one of my bete noir's for the last couple of years has been automating the quoting system. The problem is that every single inquiry really is individual and the cost of each price has to be calculated. Now the problem has been that the way of calculating a piece of stone is different for each material. I could have hard coded this into the application but then I'm stuck whenever the parameters change or I add a new material. I couldn't store the information in a relational database because of the variable nature of the structure of the data. The solution is XML. With XML I can describe the data. When I parse the data back into my application it takes account of the descirption and constructs a suitable object which presents a standard interface to the rest of the program.

I have a separate XML file for each material meaning that I can add materials at will.

Once the price is calculated it can then conform to a relational model and I store the quote in a mySQL database.

If you still feel that XML is the right tool for your job then you have to decide where to store the XML in a database or just as text files. You can write an application to parse XML files and do everything with them that you could with data in a relational db. It's just a different approach.

To get the full advantage from XML there is no point in storing it in a db and then just using SELECT statements to search it. You've got to get the data out of the db parse it in a program.

Keeping XML in text files works fine if the volume isn't too large or they are easy to find (for example if each file was an invoice, or invoices for one client just so long as you knew which client or invoice you wanted to look up then it would be easy to find the right file). If this is not the case and you decide to store it in a db then you've probably got to store some meta data so that you can easily retrieve your XML data.

At the end of the day choose the tool to fit the task - don't use something because it's hyped.

Good Luck.

Jethro

I forgor to mention. An excellent book on working with XML is 'XML Processing with Python' by Sean McGrath. The author worked on the W3C SIG that standardised XML and knows his stuff. He prefers Python above all other languages for parsing XML (no flames please).

Jethro