IWETHEY v. 0.3.0 | TODO
1,095 registered users | 0 active users | 0 LpH | Statistics
Login | Create New User
IWETHEY Banner

Welcome to IWETHEY!

New Re: In Dejavu, one would write:
> f = lambda x: x.description.startswith("hibernate") and x.amount > 100\n> for unit in recall(Thing, logic.Expression(f)):\n>     do_something_with(unit)
This looks a lot like the criteria library ([link|http://onestepback.org/index.cgi/Tech/Ruby/Criteria.rdoc|http://onestepback.o...uby/Criteria.rdoc]). Is the implementation based on similar principles?
--
-- Jim Weirich jim@weirichhouse.org [link|http://onestepback.org|http://onestepback.org]
---------------------------------------------------------------------
"Beware of bugs in the above code; I have only proved it correct,
not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas)
New An earlier version was like that.
I had a class which would absorb criteria via Python expressions, so:

e = logic.Expression()\ne.attribute == 3


would trigger:

class Expression(object):\n    def __eq__(self, other):\n        stack.append(operator.eq, other)


Etcetera. Then I would pass around the stack and evaluate it as necessary. But I also (like Criteria) had the issue of not being able to override shortcut And and Or (I used binary & and | instead), and it got a bit ugly. In addition, I was abusing normal syntax--as above, line 2, most manipulations took a comparison and didn't return or assign anything; it wasn't very natural.

So, after seeing a brilliant post by Raymond Hettinger on c.l.p., I wrote two things: an EarlyBinder and a LambdaDecompiler class. Passing a lambda (a Python anonymous function) into an Expression constructor now does early binding via a bytecode Visitor, grabbing globals and cell objects from the closure and dereferencing them (making them constants), then making a new function object. At that point, I'm free to pass it around, even make a string out of it and store it in a DB, because all variables are either arguments to the function, or constants. Flexibility of function is provided through a single mechanism; you can write:

lambda x, **kw: x.Size > kw['Size']


..where kw is keyword arguments--a dictionary (mapping; hash to you Perlers).

The LambdaDecompiler also is a bytecode Visitor, and produces the Python source code from bytecode--that's how I get a string to store. But, having that decompiler core, I subclass it and write SQLDecompilers as well. So the above decompiles into SQL Server (ADO) as:

"WHERE [Size] > %(Size)s" % kwargs


...which, when evaluated with kwargs = {'Size': 3}, yields WHERE [Size] > 3. This gets me the speed boost you mentioned (avoiding 'SQL fragments').

But it was also important for me to have an expression which I could then pass a fully-formed class and test it, because:

1) I have an in-memory object cache, and I need to filter it by the same logic, and
2) The match between Python logic and SQL is imperfect. For example, == is case-sensitive in Python but -insensitive in ADO SQL. It's fastest just to create the class object and run it through the lambda than try to wrangle SQL Server into making the search case-sensitive, which is set database-wide, per column, *and* per query IIRC.

The only ugly bit now is that it's CPython-specific, it won't work on Jython (Python which runs on the JVM), because the bytecode is different. Meh. One thing I really like is that the application code (what a developer using my framework would write) doesn't mention SQL at all. So you can use exactly the same code for huge DB tables, or 10-item in-memory lists, and leave those decisions about the best storage mechanism up to the deployers. They can decide that a list of 10 small items should stay in memory, and the huge table shouldn't. Or vice-versa.
New Forgot: code links
The logic module contains the Expression class, which wraps up the lambdas and adds some sugar, like simpler filters, and allowing you to add (logical-and) Expressions together:

>>> f = logic.Expression(lambda x: x.Size > 3)\n>>> g = logic.filter(Color='Blue')\n>>> f += g\n>>> f\nlogic.Expression(lambda x: (x.Size > 3) and (x.Color == 'Blue'))


The codewalk module does the bytecode inspection, etc.

Both at [link|http://www.aminus.org/rbre/python/index.html|http://www.aminus.or...python/index.html]
     Have I mentioned lately that Python is BITCHIN??? - (FuManChu) - (167)
         The solution is to toss OO to begin with, not Python - (tablizer) - (166)
             Re: The solution is to toss OO to begin with, not Python - (admin) - (11)
                 not today -NT - (tablizer) - (2)
                     *BLAM BLAM* *VROOOOOooooommm.....* -NT - (pwhysall) - (1)
                         and when you do the same? -NT - (tablizer)
                 Bloat explanation - (tablizer) - (7)
                     Not what I meant. - (admin)
                     That's a very table-centric point of view - (FuManChu) - (5)
                         Of course. Tables rock. Code is ugly. - (tablizer) - (4)
                             Snort. - (FuManChu) - (1)
                                 Re: Snort - (tablizer)
                             So now you're down on "skinny" tables? - (ben_tilly) - (1)
                                 Re: So now you're down on "skinny" tables? - (tablizer)
             Not sure what you're pushing for. - (FuManChu) - (153)
                 Software engineering is gambling - (tablizer) - (152)
                     The only thing this app does is business modeling. -NT - (FuManChu) - (151)
                         I meant showing me code, not anecdotes -NT - (tablizer) - (150)
                             Ironic, considering how much code *you* have shown... -NT - (ben_tilly) - (9)
                                 Wonder how the L Compiler is getting on? -NT - (pwhysall) - (6)
                                     It's Late. - (admin) - (5)
                                         I was wondering where the L it was. -NT - (pwhysall)
                                         The P game - (tablizer) - (3)
                                             Re: The P game - (admin) - (2)
                                                 you still have no evidence of betterment - (tablizer) - (1)
                                                     No, you compare it. - (admin)
                                 The burden of evidence is on you - (tablizer) - (1)
                                     And what claim did I ever make? - (ben_tilly)
                             Well, here's the thing - (FuManChu) - (139)
                                 Unicorns - (tablizer) - (138)
                                     OT: Have you been reading Groklaw tonight too? - (Another Scott)
                                     General description - (FuManChu) - (136)
                                         SQL wrapper? - (tablizer) - (135)
                                             Generic SQL wrapper API - (admin) - (30)
                                                 I see bloated people - (tablizer) - (29)
                                                     Er... - (admin) - (19)
                                                         re: Er... - (tablizer) - (18)
                                                             Nice attempt - (admin) - (17)
                                                                 re: Nice attempt - (tablizer) - (13)
                                                                     re: Nice attempt - (admin) - (12)
                                                                         Forgot to add: - (admin)
                                                                         Aieee! Actual code! HOW DARE YOU!!! -NT - (pwhysall) - (10)
                                                                             Making it interesting - (ChrisR) - (9)
                                                                                 Heh. I think Bryce would rather have someone else write it - (FuManChu)
                                                                                 Don't you know? - (Arkadiy)
                                                                                 Putting words in my mouth - (tablizer) - (6)
                                                                                     Actually, what you said was: - (admin)
                                                                                     Re: Putting words in my mouth - (admin)
                                                                                     Code Talks. Bryce Walks. - (ChrisR) - (3)
                                                                                         I shall consider it -NT - (tablizer) - (2)
                                                                                             Thanks. - (ChrisR)
                                                                                             If you don't... - (admin)
                                                                 RE: Are you willing to admit this yet? - (ChrisR) - (2)
                                                                     Even OO fans are mixed about OR-mappers - (tablizer) - (1)
                                                                         Looks pretty specific. - (admin)
                                                     In Dejavu, one would write: - (FuManChu) - (8)
                                                         Re: In Dejavu, one would write: - (JimWeirich) - (2)
                                                             An earlier version was like that. - (FuManChu) - (1)
                                                                 Forgot: code links - (FuManChu)
                                                         ICLRPD - (drewk)
                                                         Why learn and/or create another query language? - (tablizer) - (3)
                                                             In case you missed it: - (FuManChu) - (2)
                                                                 only for the trivial - (tablizer) - (1)
                                                                     There are multiple ways of doing that... - (FuManChu)
                                             LCD *for my framework* - (FuManChu) - (103)
                                                 RDBMS > "data store" - (tablizer) - (102)
                                                     Re: RDBMS > "data store" - (admin) - (92)
                                                         still have not justified yet another tool/layers - (tablizer) - (91)
                                                             Re: still have not justified yet another tool/layers - (admin) - (90)
                                                                 before and after - (tablizer) - (89)
                                                                     Re: before and after - (admin) - (88)
                                                                         Then why have an OR-mapper middleman? -NT - (tablizer) - (87)
                                                                             Because: - (admin) - (86)
                                                                                 I forgot about query caching--a big part of my other app. -NT - (FuManChu)
                                                                                 Those are problems that OO introduces - (tablizer) - (84)
                                                                                     Re: Those are problems that OO introduces - (admin) - (83)
                                                                                         response - (tablizer) - (82)
                                                                                             Re: response 1 - (admin) - (3)
                                                                                                 Looks like a lot of beurOOcracy to me - (tablizer) - (2)
                                                                                                     Not claiming "victory" - (admin)
                                                                                                     Let's see your version, then. -NT - (pwhysall)
                                                                                             Re: response 2 - (admin) - (16)
                                                                                                 Re Re: response 2 - (tablizer) - (15)
                                                                                                     HQL is a superset of SQL - (admin) - (13)
                                                                                                         re: HQL is a superset of SQL - (tablizer) - (12)
                                                                                                             re: HQL is a superset of SQL - (admin) - (11)
                                                                                                                 Forced to use A to get B? Tsk tsk tsk. How MS of them. -NT - (tablizer) - (10)
                                                                                                                     You miss the point, anyway. - (admin) - (9)
                                                                                                                         I am not necessarily disagreeing with that here - (tablizer) - (8)
                                                                                                                             Re: I am not necessarily disagreeing with that here - (admin) - (7)
                                                                                                                                 not my burden of evidence - (tablizer) - (6)
                                                                                                                                     Wrong. - (admin) - (3)
                                                                                                                                         exist != good - (tablizer) - (2)
                                                                                                                                             Existence beats Non-Existence -NT - (ChrisR)
                                                                                                                                             I've already shown you. - (admin)
                                                                                                                                     Hm. So I can make any claim and force you to prove me wrong? - (FuManChu) - (1)
                                                                                                                                         Actually, what he said was: - (admin)
                                                                                                     Persistence vs. reporting - (admin)
                                                                                             Re: response 3 - (admin) - (55)
                                                                                                 Re Re: response 3 - (tablizer) - (54)
                                                                                                     Caching - (admin) - (2)
                                                                                                         Re: caching - (tablizer) - (1)
                                                                                                             Er, no it isn't. - (admin)
                                                                                                     Serial changes - (admin) - (50)
                                                                                                         "Bound" fields - (tablizer) - (49)
                                                                                                             No, no integration. - (admin) - (48)
                                                                                                                 re: No integration - (tablizer) - (47)
                                                                                                                     re: No integration - (admin) - (46)
                                                                                                                         So you want to see an Iwethey clone? - (tablizer) - (45)
                                                                                                                             I'd prefer you answer the points first. - (admin) - (44)
                                                                                                                                 We need code, not brochure-talk, to settle this - (tablizer) - (43)
                                                                                                                                     Re: We need code, not brochure-talk, to settle this - (admin) - (42)
                                                                                                                                         Is that part of iwethey? -NT - (tablizer) - (1)
                                                                                                                                             Yes. -NT - (admin)
                                                                                                                                         questions and comments - (tablizer) - (39)
                                                                                                                                             Reading comprehension? - (ben_tilly) - (2)
                                                                                                                                                 What is your complaint? - (tablizer) - (1)
                                                                                                                                                     Why not eliminate everything that is only a bonus? - (ben_tilly)
                                                                                                                                             Then don't bother. - (admin) - (16)
                                                                                                                                                 Fine. It is not "bloated". Just unknown. - (tablizer) - (15)
                                                                                                                                                     That's all I needed. - (admin) - (8)
                                                                                                                                                         Probably not telling you anything you don't already know... - (ChrisR) - (5)
                                                                                                                                                             I take it you mean... - (admin) - (1)
                                                                                                                                                                 Yep = That's what I meant - (ChrisR)
                                                                                                                                                             Re: Probably not telling you anything you don't already know - (JimWeirich) - (2)
                                                                                                                                                                 My memory is too taxed - (ChrisR)
                                                                                                                                                                 Memory Lane - (tablizer)
                                                                                                                                                         re: That's all I needed. - (tablizer) - (1)
                                                                                                                                                             re: That's all I needed. - (admin)
                                                                                                                                                     Caching - (admin)
                                                                                                                                                     Login checks and declarative processing - (admin) - (4)
                                                                                                                                                         re: Login checks and declarative processing - (tablizer) - (3)
                                                                                                                                                             re: Login checks and declarative processing - (admin) - (2)
                                                                                                                                                                 re re: Login checks and declarative processing - (tablizer) - (1)
                                                                                                                                                                     Re: Login checks and declarative processing (new thread) - (admin)
                                                                                                                                             I18N - (admin) - (18)
                                                                                                                                                 You are WRONG! - (tablizer) - (17)
                                                                                                                                                     My my, I touched a raw nerve, apparently. - (admin) - (16)
                                                                                                                                                         Grep is no substitute for clean, normalized data - (tablizer) - (15)
                                                                                                                                                             Re: Grep is no substitute for clean, normalized data - (admin) - (14)
                                                                                                                                                                 Depends on the scale - (tablizer) - (13)
                                                                                                                                                                     You've never used CVS either, have you? -NT - (FuManChu) - (3)
                                                                                                                                                                         No, only MS sourcesafe -NT - (tablizer) - (2)
                                                                                                                                                                             *shudder* - (admin) - (1)
                                                                                                                                                                                 Not my pickings -NT - (tablizer)
                                                                                                                                                                     Re: Depends on the scale - (admin) - (8)
                                                                                                                                                                         no no no - (tablizer) - (7)
                                                                                                                                                                             Re: no no no - (admin) - (6)
                                                                                                                                                                                 Doing my part to promote right shifting - (ChrisR) - (3)
                                                                                                                                                                                     "Oh goody, a new framework for xmas!" - (tablizer) - (1)
                                                                                                                                                                                         Re: "Oh goody, a new framework for xmas!" - (admin)
                                                                                                                                                                                     Internationalization (new thread) - (admin)
                                                                                                                                                                                 admin - (tablizer) - (1)
                                                                                                                                                                                     Resource files (new thread) - (admin)
                                                                                             Re: response 4 - (admin) - (3)
                                                                                                 Lock-in is lock-in - (tablizer) - (2)
                                                                                                     Query caching is a feature of JDBC - (admin) - (1)
                                                                                                         Yes - (tablizer)
                                                                                             Re: response 5 - (admin)
                                                     Most of those are problems, not solutions. - (FuManChu) - (8)
                                                         intense DBA and RDBMS bashing - (tablizer) - (7)
                                                             ..heh. Only because they deserve it. ;) - (FuManChu) - (6)
                                                                 re: Only because they deserve it. - (tablizer) - (5)
                                                                     I assume you already expect my response: - (FuManChu) - (4)
                                                                         I will agree that... - (tablizer) - (3)
                                                                             I've never claimed that _every_ O-R mapper is chicken soup - (FuManChu) - (2)
                                                                                 Most people can at least read Python within 5 minutes. -NT - (admin) - (1)
                                                                                     Whatever -NT - (tablizer)

*plonk*
324 ms