IWETHEY v. 0.3.0 | TODO
1,095 registered users | 0 active users | 0 LpH | Statistics
Login | Create New User
IWETHEY Banner

Welcome to IWETHEY!

New need to hammer an ldap directory to its knees
currently I am using 6 different machines using slapd to authenticate against an ldap directory replica. The highest transaction rate I can get is 180 queries a second I need to get up to 600 queries a second as the directory machine is running on idle. Any ideas?
thanx,
bill
Any opinions expressed by me are mine alone, posted from my home computer, on my own time as a free american and do not reflect the opinions of any person or company that I have had professional relations with in the past 50 years. meep
New Okay, I need to know...
Memory - Scale your cache to use all (or close to it) available memory. Increase system memory if you can.

Disk - Use fast subsystems. Put databases and logs on separate disks.

Filesystem - Tune your file system. Disable access and modification timestamps to files. Enable softupdates or async access. Solaris can be hit badly by this one.

You may also want to start using "pcache" or Proxy cache to remember lotsa stuff for quick access.

Setting cached entries alot higher in the replica config (and using more memory) i.e.: cachesize 500000
That allows it to cache 500K items at a time.

Also Indexing frequently searched for items i.e.: index cn,sn,givenname,mail eq
Just remember indexes require more memory and time for maintaining them. You can have as many indexes as you see fit. That sample was just one index. You can have an index on ONE field if you so choose.

You may also want to increase the size of the DBcache size i.e.: dbcachesize 1000000
This allows 1MB for each open index file. You can go much higher, but watch your memory usage on it., if you have a ton of Indexes. Do go higher if you have only a couple of indexes or a series of multiple field indexes.

Figuring you cache size is a toughy.

A "typical" example would probably be something like this:
dn2id using 500(+1) Internal Pages using 4KB page frames/size : 501 * 4KB == 2,052,096 Bytes

id2entry using 70(+1) Internal Pages using 16KB page frames/size : 71 * 16KB == 1,163,264 Bytes

So : 2,052,096 + 1,163,264 == 3,215,360 Bytes. This is for just the DB, no indexes or other cache or overhead. But once you get the entire B-Tree in memory, there is no gain by throwing more cache at it. You will get the same speed regardless. But it would allow for growth. Also after OpenLDAP 2.2 there is no longer Hashes for the cache to worry about...

So far we have about ~3.1MB of memory for this example LDAP DB. Onto the index and object caching tasks. Indexes also use B-Tree setup. Pretty much if you cannot hold 100% of the data in memory then using a percentage of it is all that is needed. For each index, there is a seperate file. Let us say we have a one object index, being 7MB in size for this example. A good thing to figure is 20%-25% of the size of the index size or in this case 1,651,507 Bytes for the threshold.

So right now the amount of Bare Minimum is: 2,052,096 + 1,163,264 + 1,651,507 == 4,866,867

so ~4.5MB would be enough for this example. This does not account for library compile time options increasing thing, depending on your compile options. This also does not account for any overhead needed for things related to OpenLDAP's self-keeping needs.


You can find out most if not all of this info (the Internal Pages counts, etc...) with db_stat (or db4.2_stat in Ubuntu's Dapper) and db_stat -m (etc... do the man db_stat thing)

Also, to hammer a server to see what it can do... you might consider looking at this script:[link|http://www.opendarwin.org/~jtownsend/bindstress.pl|bindstress.pl] you just need to edit the script to match your ldap server. Then run it like 1-20 times concurrnetly to find out what you need to do. You can run this from multiple machines at the same time.

You might also look at some of the other things he has done/published/worked on: [link|http://www.opendarwin.org/~jtownsend/|http://www.opendarwin.org/~jtownsend/]
--
[link|mailto:greg@gregfolkert.net|greg],
[link|http://www.iwethey.org/ed_curry|REMEMBER ED CURRY!] @ iwethey
Freedom is not FREE.
Yeah, but 10s of Trillions of US Dollars?
SELECT * FROM scog WHERE ethics > 0;

0 rows returned.
New Im tuned to the nth, thats not the problem
problem is I cant hit it hard enough to matter, its not the server its the clients not pushing enough connections. Im trying to get a base line on a production 280r then hammer a T2000 with the same load and see the preformance difference. Will try the bindstress.pl thanks for the tips.
thanx,
bill
Any opinions expressed by me are mine alone, posted from my home computer, on my own time as a free american and do not reflect the opinions of any person or company that I have had professional relations with in the past 50 years. meep
     need to hammer an ldap directory to its knees - (boxley) - (2)
         Okay, I need to know... - (folkert) - (1)
             Im tuned to the nth, thats not the problem - (boxley)

Many died, but in the end, the chickens won.
107 ms