In Dejavu, one would write:

Post #156,191 by FuManChu 5/21/04 3:27:29 AM Reply	In Dejavu, one would write: f = lambda x: x.description.startswith("hibernate") and x.amount > 100\nfor unit in recall(Thing, logic.Expression(f)):\n do_something_with(unit) instead of: sql = 'select * from bidView where description like "hibernate%" and amount > 100'\nfor row in db.execute(sql, connection):\n do_something_with(row) ...I don't see any bloat. On the contrary, the first one is written in the same programming language throughout. The second one introduces a declarative language (SQL) inside the original application language. And it only applies to databases where Like is a valid operator, and where the percent-sign is a wildcard character, and where double-quotes are valid delimiters, and where keywords (like 'select') can be lowercase, and where bidView isn't forced into a namespace (with a prefix), and where 'amount' and 'description' are not reserved words. In addition, user input is made far more safe--you can't simply insert "; DELETE * FROM important_table" into Dejavu code. ;)
Post #156,196 by JimWeirich 5/21/04 7:03:27 AM Reply	Re: In Dejavu, one would write: > f = lambda x: x.description.startswith("hibernate") and x.amount > 100\n> for unit in recall(Thing, logic.Expression(f)):\n> do_something_with(unit) This looks a lot like the criteria library ([link\|http://onestepback.org/index.cgi/Tech/Ruby/Criteria.rdoc\|http://onestepback.o...uby/Criteria.rdoc]). Is the implementation based on similar principles? -- -- Jim Weirich jim@weirichhouse.org [link\|http://onestepback.org\|http://onestepback.org] --------------------------------------------------------------------- "Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas)
Post #156,215 by FuManChu 5/21/04 11:58:02 AM Reply	An earlier version was like that. I had a class which would absorb criteria via Python expressions, so: e = logic.Expression()\ne.attribute == 3 would trigger: class Expression(object):\n def __eq__(self, other):\n stack.append(operator.eq, other) Etcetera. Then I would pass around the stack and evaluate it as necessary. But I also (like Criteria) had the issue of not being able to override shortcut And and Or (I used binary & and \| instead), and it got a bit ugly. In addition, I was abusing normal syntax--as above, line 2, most manipulations took a comparison and didn't return or assign anything; it wasn't very natural. So, after seeing a brilliant post by Raymond Hettinger on c.l.p., I wrote two things: an EarlyBinder and a LambdaDecompiler class. Passing a lambda (a Python anonymous function) into an Expression constructor now does early binding via a bytecode Visitor, grabbing globals and cell objects from the closure and dereferencing them (making them constants), then making a new function object. At that point, I'm free to pass it around, even make a string out of it and store it in a DB, because all variables are either arguments to the function, or constants. Flexibility of function is provided through a single mechanism; you can write: lambda x, *kw: x.Size > kw['Size'] ..where kw is keyword arguments--a dictionary (mapping; hash to you Perlers). The LambdaDecompiler also is a bytecode Visitor, and produces the Python source code from bytecode--that's how I get a string to store. But, having that decompiler core, I subclass it and write SQLDecompilers as well. So the above decompiles into SQL Server (ADO) as: "WHERE [Size] > %(Size)s" % kwargs ...which, when evaluated with `kwargs = {'Size': 3}`, yields `WHERE [Size] > 3`. This gets me the speed boost you mentioned (avoiding 'SQL fragments'). But it was also important for me to have an expression which I could then pass a fully-formed class and test it, because: 1) I have an in-memory object cache, and I need to filter it by the same logic, and 2) The match between Python logic and SQL is imperfect. For example, == is case-sensitive in Python but -insensitive in ADO SQL. It's fastest just to create the class object and run it through the lambda than try to wrangle SQL Server into making the search case-sensitive, which is set database-wide, per column, and* per query IIRC. The only ugly bit now is that it's CPython-specific, it won't work on Jython (Python which runs on the JVM), because the bytecode is different. Meh. One thing I really like is that the application code (what a developer using my framework would write) doesn't mention SQL at all. So you can use exactly the same code for huge DB tables, or 10-item in-memory lists, and leave those decisions about the best storage mechanism up to the deployers. They can decide that a list of 10 small items should stay in memory, and the huge table shouldn't. Or vice-versa.
Post #156,218 by FuManChu 5/21/04 12:42:02 PM Reply	Forgot: code links The logic module contains the Expression class, which wraps up the lambdas and adds some sugar, like simpler filters, and allowing you to add (logical-and) Expressions together: >>> f = logic.Expression(lambda x: x.Size > 3)\n>>> g = logic.filter(Color='Blue')\n>>> f += g\n>>> f\nlogic.Expression(lambda x: (x.Size > 3) and (x.Color == 'Blue')) The codewalk module does the bytecode inspection, etc. Both at [link\|http://www.aminus.org/rbre/python/index.html\|http://www.aminus.or...python/index.html]
Post #156,212 by drewk 5/21/04 11:09:34 AM Reply	ICLRPD It only applies to databases where Like is a valid operator, and where the percent-sign is a wildcard character, and where double-quotes are valid delimiters, and where keywords (like 'select') can be lowercase, and where bidView isn't forced into a namespace (with a prefix), and where 'amount' and 'description' are not reserved words. === Implicitly condoning stupidity since 2001.
Post #156,314 by tablizer 5/22/04 1:52:44 AM Reply	Why learn and/or create another query language? SQL is fairly well known. Why introduce yet another query language for developers to learn? Why not come up with a generic SQL dialect and then translate it into different dialects instead of a very different query language into SQL dialects? Personally I see that most places don't swap DB vendors that often so it is not worth worrying about. I would like to toss SQL also for something easier to standardize, but your approach is not it. BTW description.startswith("hibernate") is much more cluttery than like "hibernate%" One could make string processing functions to do something similar if need be, I would note. At least get rid of that damned demeter-violating dot bloat and make it just "startsWith()". ________________ oop.ismad.com
Post #156,339 by FuManChu 5/22/04 10:55:32 AM Reply	In case you missed it: as I said, I am not introducing another query language. The language which you are viewing as "another query language" is nothing more nor less than pure Python, and it applies perfectly to the fully-formed Python objects with which I wish to work. For example, with the expression: f = lambda x: x.description.startswith("hibernate") I have just defined an anonymous function in Python. I can then use it directly, without any further translation, on a Thing object: class Thing(object):\n def __init__(self, description):\n self.description = description\n\nbear = Thing("hibernates in winter") And `f(bear)` returns True without any intermediate magic. I hope you're not saying that calling a function is more than a programmer can handle. I will concede the fact that lambdas seem to mystify some people. Given that simplicity, since functions are first-class objects in Python, I can pass that function down to a server like so: `hibernators = namespace.recall(Thing, f)` ...and get back a list of all Thing objects for which the function `f` returns True. Second point you might have missed: not every storage mechanism is going to be SQL-based. Some might be better off without persistence altogether. Some might be better off with a bsddb. So my solution is even more generic than a "generic SQL dialect". Third, don't cheat. You wrote: description.startswith("hibernate") is much more cluttery than like "hibernate%" ...conveniently leaving out the field name in your example to make it shorter. At the least, you should have written: description.startswith("hibernate") is much more cluttery than description like "hibernate%" I won't digress again into how UN-generic and cluttery "like" and "%" are. But I will note that new developers are far more likely to grok my Python syntax at first glance than your SQL.
Post #156,363 by tablizer 5/22/04 1:48:38 PM Reply	only for the trivial conveniently leaving out the field name in your example to make it shorter. I assure you it was not intentional. I apologize for the error. But I will note that new developers are far more likely to grok my Python syntax at first glance than your SQL. More developers know SQL than know Python. And I am still skeptical it can handle non-trivial queries. BTW, How would your framework handle this: \n SELECT AVG(GPA) FROM (\n\tSELECT AVG(grade) AS GPA FROM Grades GROUP BY studentID\n )\n It calculates total GPA with an equal weight per student rather than per class. ________________ oop.ismad.com
Post #156,414 by FuManChu 5/22/04 6:09:07 PM Reply	There are multiple ways of doing that... ...depends to a large extent on how the designer puts things together. If we had, for example, a Grade Unit and a Student Unit: class Grade(dejavu.Unit):\n pass\nGrade.set_properties({'Value': float,\n 'StudentID': unicode})\n\nclass Student(dejavu.Unit):\n def GPA(self):\n grades = [x.Value for x in self.Grade()]\n if grades:\n return sum(grades) / len(grades)\n else:\n return 0.0 # This may not be what you want\n\nnamespace.associate(Student, 'ID', Grade, 'StudentID') That .associate function binds a new .Grade() method to the Student class; said method looks up all Grade objects which have a StudentID equal to our Student.ID; storage managers are free to optimize that call with a JOIN if they manage both classes. Our Student.GPA() function uses this (as "self.Grade()") to recall associated units. So developer code to calc the GPA for multiple students would then look like this (assuming you want all students, all years, etc.): gpas = [x.GPA() for x in namespace.recall(Student)]\nif gpas:\n avg_gpa = sum(gpas) / len(gpas)\nelse:\n avg_gpas = 0.0 # Again, this may not satisfy requirements... Is it as fast as straight SQL? Guaranteed not. But is it fast enough? Probably. If it isn't, there's nothing stopping the developer from hand-crafting an SQL statement and bypassing some of the layers, if he wants to completely sacrifice maintainability and portability. But I've built three apps on top of this framework now, and haven't run into a speed conflict yet. Fu xiansheng And, incidentally, makes a reciprocal Student() method for the Grade class. ** However, one of the design points of Dejavu is that the framework does not assume all data is managed by the same storage manager unless explicitly told so by the deployer. At my company, for example, I have expense Transactions in one store and income in another, and have to mix them to show, say, amount paid on an invoice. This is done transparently from the point of view of application code--it's handled by a mixing Unit Server in the middle layer. When the day comes that we decide to stop using the second store (which is 3rd party code), all I have to do is change a line in a .conf file, and roll on. I could even build a tool very quickly to migrate the data for me; it's on my list of things to write for version 4.1.

Welcome to IWETHEY!