IWETHEY v. 0.3.0 | TODO
1,095 registered users | 0 active users | 0 LpH | Statistics
Login | Create New User
IWETHEY Banner

Welcome to IWETHEY!

New A question you might ask yourself
and I'm asking more and more is "do you really need a database"? Could you do as well with a well thought out directory structure and files? More and more I'm finding the answer is "probably".

Of course, I've become a bit more adventurous since working at big river. There are precious few conventional databases to be found there, but lots of clever data stores.




[link|http://www.blackbagops.net|Black Bag Operations Log]

[link|http://www.objectiveclips.com|Artificial Intelligence]

[link|http://www.badpage.info/seaside/html|Scrutinizer]
New It is a promising thought, I'll admit.
I've suggested something like that to my IT colleagues, but I haven't so much as hinted it at the web developers yet; they'll challenge it unless it can be given to them working and better than what is in place now. And I really need them using the API before I can do anything like that. At the moment, there is a still a lot of SQL done in the website...

I do empathise. I see signs of 'it's a database - that must be the way to do it' :-/

Wade.
"Don't give up!"
New Just gone through that
I've automated some inbound email->order entry into one of our systems.
I'm not supposed to create an order for certain type of products when the inventory is low, I'm supposed to hold the order for the next day's processing and try again.

So of course, hold order == database item. Or so I thought.

Than I realized the inbound email had to be stored in it's entirety, which really means an index value of inbound sequence number, and then a blob. I needed to parse it new each time since the parsing took into account external translation tables, which could change each run, which means there is no win to storing the post-parsed message.

Things are never on hold for very long, and the qty of on-held orders is low, max into the hundreds.

So, I simply created a hold file for each message and feed them back into my process every day. The file name is the inbound timestamp which insures FIFO processing.
New Are we going back to frame-style usage?
What you're talking about sounds like the way I understand mainframe "databases" work. I may be making connections that aren't there, but it sounds like there was a style that worked well on the frame. Then we developed relational databases. They were a convenient abstraction for the programmers. They worked well for smaller problems and smaller datasets, and ran on midranges and workstations.

Then as the available horsepower grew some of the old frame jobs were re-written to use relational DBs on fast workstations. (Yes, Barry, this means you.) Now it sounds like the problems we're trying to solve -- the size of data and the number of transactions -- are exceeding what we can do well with relational DBs. So now people are starting to re-invent the style of creating their own domain-specific database.

Is this completely wrong?
===

Kip Hawley is still an idiot.

===

Purveyor of Doc Hope's [link|http://DocHope.com|fresh-baked dog biscuits and pet treats].
[link|http://DocHope.com|http://DocHope.com]
New Interesting take on things.
Has some merit, I'll admit. I thought I liked the idea of having one MySQL database tuned for some of my data and another tuned differently for another lot of data. You're suggesting that flatfiles might be better for some other types of data and perhaps some other, highly custom arrangement better for another sub-domain.

But then, you couldn't do the sort of adhoc stuff on mainframe datasets that we do now on many relational DBs. You had to program something and wait for it to finish. Which might be several minutes, it might be several hours, it might be several days.

I think the role of 'database programmer' is being re-invented.

Wade.
"Don't give up!"
New Best tool for the job and all that
People treat SQL databases as golden bullets for their data storage needs, rarely taking into account the downsides. Time to take a step back and figure out what you really need to accomplish, and THEN determine the best method of storage and access.
New Yah
Too many people just think "I need to store a lot of data on disk - I guess I need a database". They don't realize that databases COST you something too.

If its just about persistence - probably you can do better with plain old files. OTOH, if you really do have a lot of different ways of looking at your data, then a database is just the thing.

I'm actually moving an app from an oodb, which is mostly only good at persistence, to postgres because I have ever expanding reporting requirements.




[link|http://www.blackbagops.net|Black Bag Operations Log]

[link|http://www.objectiveclips.com|Artificial Intelligence]

[link|http://www.badpage.info/seaside/html|Scrutinizer]
New Power of abstractions vs. absolute performance
OODBMSs are supposed to offer better abstractions to make programming easier, but I haven't heard anyone say the performance is acceptable. Eventually the hardware will pick up enough that today's experiments will become practical. But by then we'll be trying to do more, with more data.

Maybe we should keep the greybeards around so that when we hit the wall of what we can do with current hardware, they can remind us how they used to do things.
===

Kip Hawley is still an idiot.

===

Purveyor of Doc Hope's [link|http://DocHope.com|fresh-baked dog biscuits and pet treats].
[link|http://DocHope.com|http://DocHope.com]
     Okay, I'll relent. (edited/updated) - (folkert) - (23)
         Take a compiler course - (jake123)
         My favorite books - (ChrisR) - (18)
             New compiler books needed? - (tonytib) - (17)
                 A new dragon book was put out this year - (ChrisR)
                 I'd like one about building scalability layers. - (static) - (14)
                     Scalability layer ... - (drewk) - (11)
                         It's farily straightforward. - (static) - (10)
                             What's that buzzing sound? - (drewk) - (9)
                                 Not so fast, bubba boy. - (static) - (8)
                                     A question you might ask yourself - (tuberculosis) - (7)
                                         It is a promising thought, I'll admit. - (static)
                                         Just gone through that - (crazy)
                                         Are we going back to frame-style usage? - (drewk) - (4)
                                             Interesting take on things. - (static)
                                             Best tool for the job and all that - (crazy) - (2)
                                                 Yah - (tuberculosis) - (1)
                                                     Power of abstractions vs. absolute performance - (drewk)
                     Taking SQL away - (tuberculosis) - (1)
                         Interesting. - (static)
                 Related but a bit OT question - (tjsinclair)
         What about embedded development? - (tonytib) - (2)
             Would any of these - (tuberculosis) - (1)
                 Possibly, but would require more hardware - (tonytib)

According to one source, the average British driver will let rip 912 pints of fart during their lifetime.
93 ms