[gridengine users] qmaster: hard descriptor limit and soft descriptor limit?

Discussion:

Brodie, Kent

2012-10-12 18:55:07 UTC

Reuti

2012-10-12 19:40:03 UTC

Permalink

It is set during boot when the daemon is started (the limit can be different from a later user login). But you can check the parameters:

S_DESCRIPTORS, H_DESCRIPTORS

in `man sge_conf`.

-- Reuti

Post by Brodie, Kent
I have 8192 (soft) and 65536 (hard) set for my system-wide file descriptor limits .
....
open files (-n) 8192
...
open files (-n) 65536
Hopefully missing something obvious, --Kent
_______________________________________________
users mailing list
https://gridengine.org/mailman/listinfo/users

Brodie, Kent

2012-10-12 20:30:49 UTC

Permalink

I'm confused. I thought that parameter relates to execution hosts? (for which, I have confirmed it's correct). AND- I have my execd params set as follows:

execd_params S_DESCRIPTORS=8192,H_DESCRIPTORS=65536

I'm referring to the QMASTER message from the qmaster spool. I am concerned about 1024 being too low.

Brodie, Kent

2012-10-12 20:51:53 UTC

Permalink

Nevermind. I have it working somehow. I must have set those S_ and H_ descriptor settings after the qmaster started. *Swore* I rebooted everything, but who knows. I restarted qmaster and everything looks fine. Happy weekend, everyone.

Brodie, Kent

2012-10-12 20:59:26 UTC

Permalink

Alex Chekholko

2012-10-12 21:19:31 UTC

Permalink

I think those env settings are inherited from the parent process. So
maybe qmaster is started up by some init process that didn't get its
settings from limits.conf?

I bet if you restart the process, it'll get the settings from your
current root shell, and all will be well.

Regards,

--
Alex Chekholko ***@stanford.edu

Brodie, Kent

2012-10-12 21:26:03 UTC

Permalink

Yes, this is the case. Restarting (qmaster) again, it's fine. (8192 / 65536)

OK, so how the heck do I get my qmaster process to have the right number of file descriptors following a reboot? I really can't have this so I have to manually re-restart the qmaster. And I'd be floored to discover I'm the first to see this.....

Between having the increased numbers in both /etc/sysctl.conf and /etc/security/limits.conf, I've followed any redhat document I can find.

Obviously, something is missing. I agree, it has to do with the init process.

Post by Brodie, Kent
WHOA. Where the heck is the 1024 limit coming from?

I think those env settings are inherited from the parent process. So maybe qmaster is started up by some init process that didn't get its settings from limits.conf?
I bet if you restart the process, it'll get the settings from your current root shell, and all will be well.
Regards,
--

_______________________________________________

Brodie, Kent

2012-10-12 21:52:49 UTC

Permalink

Got around by editing /etc/init.d/sgemaster.{cellname} , and adding "ulimit -n 8192" in there near the top.

If anyone has a more elegant solution, I'd love to know it!

Happy weekend everyone.

Dave Love

2012-10-15 10:47:59 UTC

Permalink

Post by Reuti
S_DESCRIPTORS, H_DESCRIPTORS
in `man sge_conf`.

To avoid possible confusion: those are execd parameters. MAX_DYN_EC is
the relevant qmaster one, limited by the process file descriptor limit
and a number set aside for static connexions. (It was raised a while
ago from a much lower value which only allowed a few clients.)

--
Community Grid Engine: http://arc.liv.ac.uk/SGE/

Dave Love

2012-10-15 10:44:38 UTC

Permalink

That's normal on GNU/Linux systems:

# grep files /proc/$(pgrep qmaster)/limits
Max open files 1024 1024 files

Probably I should take the default MAX_DYN_EC down to avoid the harmless
message.

Do you really need >~1000 dynamic clients?

--
Community Grid Engine: http://arc.liv.ac.uk/SGE/