[linux] How to increase Neo4j's maximum file open limit (ulimit) in Ubuntu?

Currently ulimit -n shows 10000. I want to increase it to 40000. I've edited "/etc/sysctl.conf" and put fs.file-max=40000. I've also edited /etc/security/limits.conf and updated hard and soft values. But still ulimit shows 10000. After making all these changes I rebooted my laptop. I've access to root password.

usr_name@usr_name-lap:/etc$ /sbin/sysctl fs.file-max
fs.file-max = 500000

Added following lines in /etc/security/limits.conf -

*     soft    nofile          40000
*     hard    nofile          40000

I also added following line in /etc/pam.d/su-

session    required   pam_limits.so

I've tried every possible way as given on other forums, but I can reach up to a maximum limit of 10000, not beyond that. What can be the issue?

I'm making this change because neo4j throws maximum open file limits reached error.

This question is related to linux ubuntu neo4j limit ulimit

The answer is


1) Check sysctl file-max limit:

$ cat /proc/sys/fs/file-max

If the limit is lower than your desired value, open the sysctl.conf and add this line at the end of file:

fs.file-max = 65536

Finally, apply sysctl limits:

$ sysctl -p 

2) Edit /etc/security/limits.conf and add below the mentioned

* soft     nproc          65535    
* hard     nproc          65535   
* soft     nofile         65535   
* hard     nofile         65535

These limits won't apply for root user, if you want to change root limits you have to do that explicitly:

root soft     nofile         65535   
root hard     nofile         65535
...

3) Reboot system or add following line to the end of /etc/pam.d/common-session:

session required pam_limits.so

Logout and login again.

4) Check soft limits:

$ ulimit -a

and hard limits:

$ ulimit -Ha
....

open files                      (-n) 65535

Reference : http://ithubinfo.blogspot.in/2013/07/how-to-increase-ulimit-open-file-and.html


I did it like this

echo "NEO4J_ULIMIT_NOFILE=50000" >> neo4j
mv neo4j /etc/default/

What you are doing will not work for root user. Maybe you are running your services as root and hence you don't get to see the change.

To increase the ulimit for root user you should replace the * by root. * does not apply for root user. Rest is the same as you did. I will re-quote it here.

Add the following lines to the file: /etc/security/limits.conf

root soft  nofile 40000

root hard  nofile 40000

And then add following line in the file: /etc/pam.d/common-session

session required pam_limits.so

This will update the ulimit for root user. As mentioned in comments, you may don't even have to reboot to see the change.


Try run this command it will create a *_limits.conf file under /etc/security/limits.d

echo "* soft nofile 102400" > /etc/security/limits.d/*_limits.conf && echo "* hard nofile 102400" >> /etc/security/limits.d/*_limits.conf

Just exit from terminal and login again and verify by ulimit -n it will set for * users


You could alter the init script for neo4j to do a ulimit -n 40000 before running neo4j.

However, I can't help but feel you are barking up the wrong tree. Does neo4j legitimately need more than 10,000 open file descriptors? This sounds very much like a bug in neo4j or the way you are using it. I would try to address that.


ULIMIT configuration:

  1. Login by root
  2. vi security/limits.conf
  3. Make Below entry

    Ulimit configuration start for website user

    website   soft   nofile    8192
    website   hard   nofile    8192
    website   soft   nproc    4096
    website   hard   nproc    8192
    website   soft   core    unlimited
    website   hard   core    unlimited
    
  4. Make Below entry for ALL USER

    Ulimit configuration for every user

    *   soft   nofile    8192
    *   hard   nofile    8192
    *   soft   nproc    4096
    *   hard   nproc    8192
    *   soft   core    unlimited
    *   hard   core    unlimited
    
  5. After modifying the file, user need to logoff and login again to see the new values.


I have lots of trouble getting this to work.

Using the following allows you to update it regardless of your user permission.

sudo sysctl -w fs.inotify.max_user_watches=100000

Edit

Just saw this from another user also on another stackexchange site (both work, but this version permanently updates the system setting, rather than temporarily):

echo fs.inotify.max_user_watches=100000 | sudo tee -a /etc/sysctl.conf; 
sudo sysctl -p

I was having the same issue, and got it to work by adding entries to /etc/security/limits.d/90-somefile.conf. Note that in order to see the limits working, I had to log out completely from the ssh session, and then log back in.

I wanted to set the limit for a specific user that runs a service, but it seems that I was getting the limit that was set for the user I was logging in as. Here's an example to show how the ulimit is set based on authenticated user, and not the effective user:

$ sudo cat /etc/security/limits.d/90-nofiles.conf
loginuser    soft    nofile   10240
loginuser    hard    nofile   10240
root         soft    nofile   10241
root         hard    nofile   10241
serviceuser  soft    nofile   10242
serviceuser  hard    nofile   10242

$ whoami
loginuser
$ ulimit -n
10240
$ sudo -i
# ulimit -n
10240    # loginuser's limit
# su - serviceuser
$ ulimit -n
10240    # still loginuser's limit.

You can use an * to specify an increase for all users. If I restart the service as the user I logged in, and add ulimit -n to the init script, I see that the initial login user's limits are in place. I have not had a chance to verify which user's limits are used during a system boot or of determining what the actual nofile limit is of the service I am running (which is started with start-stop-daemon).

There's 2 approaches that are working for now:

  1. add a ulimit adjustment to the init script, just before start-stop-daemon.
  2. wildcard or more extensive ulimit settings in the security file.

tl;dr set both the soft and hard limits

I'm sure it's working as intended but I'll add it here just in case. For completeness the limit is set here (see below for syntax): /etc/security/limits.conf

some_user       soft    nofile          60000
some_user       hard    nofile          60000

and activated with the following in /etc/pam.d/common-session:

session required pam_limits.so

If you set only the hard limit, ulimit -a will show the default (1024): If you set only the soft the limit ulimit -a will show (4096)

If you set them both ulimit -a will show the soft limit (up to the hard limit of course)


Examples related to linux

grep's at sign caught as whitespace How to prevent Google Colab from disconnecting? "E: Unable to locate package python-pip" on Ubuntu 18.04 How to upgrade Python version to 3.7? Install Qt on Ubuntu Get first line of a shell command's output Cannot connect to the Docker daemon at unix:/var/run/docker.sock. Is the docker daemon running? Run bash command on jenkins pipeline How to uninstall an older PHP version from centOS7 How to update-alternatives to Python 3 without breaking apt?

Examples related to ubuntu

grep's at sign caught as whitespace "E: Unable to locate package python-pip" on Ubuntu 18.04 How to Install pip for python 3.7 on Ubuntu 18? "Repository does not have a release file" error ping: google.com: Temporary failure in name resolution How to install JDK 11 under Ubuntu? How to upgrade Python version to 3.7? Issue in installing php7.2-mcrypt Install Qt on Ubuntu Failed to start mongod.service: Unit mongod.service not found

Examples related to neo4j

How to increase Neo4j's maximum file open limit (ulimit) in Ubuntu? Delete all nodes and relationships in neo4j 1.8

Examples related to limit

Laravel Eloquent limit and offset How to increase Neo4j's maximum file open limit (ulimit) in Ubuntu? Pagination using MySQL LIMIT, OFFSET How to fix the "508 Resource Limit is reached" error in WordPress? How to limit the number of dropzone.js files uploaded? Google drive limit number of download How can I list (ls) the 5 last modified files in a directory? Limiting Python input strings to certain characters and lengths MySQL JOIN with LIMIT 1 on joined table MySQL - UPDATE query with LIMIT

Examples related to ulimit

how to set ulimit / file descriptor on docker container the image tag is phusion/baseimage-docker How to increase Neo4j's maximum file open limit (ulimit) in Ubuntu? What does "ulimit -s unlimited" do?