Skip to content

Too many open files (many image on readme) #19463

Closed
@ledlamp

Description

@ledlamp

Description

A repository with 500 small images in its readme, fails to load a majority of the images with 404 and 500 errors: https://gitea.moe/lamp/500-image

Gitea logs various "too many open file" errors such as the following:
[E] RepoAssignment Invalid repo /srv/gitea/data/repositories/lamp/500-image.git: open /dev/null: too many open files
[E] RepoAssignment Invalid repo /srv/gitea/data/repositories/lamp/500-image.git: pipe2: too many open files
[E] RepoAssignment Invalid repo /srv/gitea/data/repositories/lamp/500-image.git: fork/exec /usr/bin/git: too many open files
[E] PANIC: session(release): open /srv/gitea/data/sessions/8/6/86b2632f34bf4a7e: too many open files
[I] http: Accept error: accept unix /srv/gitea/gitea.sock: accept4: too many open files; retrying in 5ms

This doesn't reproduce on the demo server: https://try.gitea.io/Lamp/500-image
so this probably depends on the environment, but i'm posting as an issue because I suspect gitea is opening more files than it ought to be.

FWIW, with fastcgi transport this issue renders the entire web server unresponsive after loading the repository. I had to change to http.

Gitea Version

1.16.6

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

regular: https://gist.githubusercontent.com/asdfisdffdg/506346f12b672abf8f8a6c35d6e71aac/raw/3dc56de93efd60dd47d37131b5abaff4fbd33e53/gistfile1.txt debug: https://gist.githubusercontent.com/asdfisdffdg/ba829b9f7451eec87a6f0da72e280f00/raw/94fe21537c152b2d248c54a3ecba586d3840134e/gistfile1.txt

Screenshots

2022-04-21_13-01-11-947 lamp500-image_-500-image-gitea moe-_Google_Ch

Git Version

1.16.6

Operating System

Rocky Linux 8

How are you running Gitea?

Using your binary

systemd service

[Unit]
Description=Gitea git server
After=network.target

[Service]
User=git
Group=git
WorkingDirectory=/srv/gitea
ExecStart=/srv/gitea/gitea web
MemoryHigh=2G
MemoryMax=3G

[Install]
WantedBy=multi-user.target

systemctl show

Type=simple
Restart=no
NotifyAccess=none
RestartUSec=100ms
TimeoutStartUSec=1min 30s
TimeoutStopUSec=1min 30s
RuntimeMaxUSec=infinity
WatchdogUSec=0
WatchdogTimestamp=Thu 2022-04-21 14:55:31 CDT
WatchdogTimestampMonotonic=2381616958832
PermissionsStartOnly=no
RootDirectoryStartOnly=no
RemainAfterExit=no
GuessMainPID=yes
MainPID=1375444
ControlPID=0
FileDescriptorStoreMax=0
NFileDescriptorStore=0
StatusErrno=0
Result=success
UID=949
GID=946
NRestarts=0
ExecMainStartTimestamp=Thu 2022-04-21 14:55:31 CDT
ExecMainStartTimestampMonotonic=2381616958770
ExecMainExitTimestampMonotonic=0
ExecMainPID=1375444
ExecMainCode=0
ExecMainStatus=0
ExecStart={ path=/srv/gitea/gitea ; argv[]=/srv/gitea/gitea web ; ignore_errors=no ; start_time=[Thu 2022-04-21 14:55:31 CDT] ; stop_time=[n/a] ; pid=1375444 ; code=(null) ; status=0/0 }
Slice=system.slice
ControlGroup=/system.slice/gitea.service
MemoryCurrent=206057472
CPUUsageNSec=[not set]
EffectiveCPUs=
EffectiveMemoryNodes=
TasksCurrent=143
IPIngressBytes=18446744073709551615
IPIngressPackets=18446744073709551615
IPEgressBytes=18446744073709551615
IPEgressPackets=18446744073709551615
Delegate=no
CPUAccounting=no
CPUWeight=[not set]
StartupCPUWeight=[not set]
CPUShares=[not set]
StartupCPUShares=[not set]
CPUQuotaPerSecUSec=infinity
CPUQuotaPeriodUSec=infinity
AllowedCPUs=
AllowedMemoryNodes=
IOAccounting=no
IOWeight=[not set]
StartupIOWeight=[not set]
BlockIOAccounting=no
BlockIOWeight=[not set]
StartupBlockIOWeight=[not set]
MemoryAccounting=yes
DefaultMemoryLow=0
DefaultMemoryMin=0
MemoryMin=0
MemoryLow=0
MemoryHigh=2147483648
MemoryMax=3221225472
MemorySwapMax=infinity
MemoryLimit=infinity
DevicePolicy=auto
TasksAccounting=yes
TasksMax=49375
IPAccounting=no
UMask=0022
LimitCPU=infinity
LimitCPUSoft=infinity
LimitFSIZE=infinity
LimitFSIZESoft=infinity
LimitDATA=infinity
LimitDATASoft=infinity
LimitSTACK=infinity
LimitSTACKSoft=8388608
LimitCORE=infinity
LimitCORESoft=0
LimitRSS=infinity
LimitRSSSoft=infinity
LimitNOFILE=262144
LimitNOFILESoft=1024
LimitAS=infinity
LimitASSoft=infinity
LimitNPROC=30859
LimitNPROCSoft=30859
LimitMEMLOCK=65536
LimitMEMLOCKSoft=65536
LimitLOCKS=infinity
LimitLOCKSSoft=infinity
LimitSIGPENDING=30859
LimitSIGPENDINGSoft=30859
LimitMSGQUEUE=819200
LimitMSGQUEUESoft=819200
LimitNICE=0
LimitNICESoft=0
LimitRTPRIO=0
LimitRTPRIOSoft=0
LimitRTTIME=infinity
LimitRTTIMESoft=infinity
WorkingDirectory=/srv/gitea
OOMScoreAdjust=0
Nice=0
IOSchedulingClass=0
IOSchedulingPriority=0
CPUSchedulingPolicy=0
CPUSchedulingPriority=0
CPUAffinity=
CPUAffinityFromNUMA=no
NUMAPolicy=n/a
NUMAMask=
TimerSlackNSec=50000
CPUSchedulingResetOnFork=no
NonBlocking=no
StandardInput=null
StandardInputData=
StandardOutput=journal
StandardError=inherit
TTYReset=no
TTYVHangup=no
TTYVTDisallocate=no
SyslogPriority=30
SyslogLevelPrefix=yes
SyslogLevel=6
SyslogFacility=3
LogLevelMax=-1
LogRateLimitIntervalUSec=0
LogRateLimitBurst=0
SecureBits=0
CapabilityBoundingSet=cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read cap_perfmon cap_bpf
AmbientCapabilities=
User=git
Group=git
DynamicUser=no
RemoveIPC=no
MountFlags=
PrivateTmp=no
PrivateDevices=no
ProtectKernelTunables=no
ProtectKernelModules=no
ProtectControlGroups=no
PrivateNetwork=no
PrivateUsers=no
PrivateMounts=no
ProtectHome=no
ProtectSystem=no
SameProcessGroup=no
UtmpMode=init
IgnoreSIGPIPE=yes
NoNewPrivileges=no
SystemCallErrorNumber=0
LockPersonality=no
RuntimeDirectoryPreserve=no
RuntimeDirectoryMode=0755
StateDirectoryMode=0755
CacheDirectoryMode=0755
LogsDirectoryMode=0755
ConfigurationDirectoryMode=0755
MemoryDenyWriteExecute=no
RestrictRealtime=no
RestrictSUIDSGID=no
RestrictNamespaces=no
MountAPIVFS=no
KeyringMode=private
KillMode=control-group
KillSignal=15
SendSIGKILL=yes
SendSIGHUP=no
Id=gitea.service
Names=gitea.service
Requires=-.mount system.slice sysinit.target
WantedBy=multi-user.target
Conflicts=shutdown.target
Before=multi-user.target shutdown.target
After=systemd-journald.socket sysinit.target basic.target network.target -.mount system.slice
RequiresMountsFor=/srv/gitea
Description=Gitea git server
LoadState=loaded
ActiveState=active
FreezerState=running
SubState=running
FragmentPath=/srv/gitea/gitea.service
UnitFileState=enabled
UnitFilePreset=disabled
StateChangeTimestamp=Thu 2022-04-21 14:55:31 CDT
StateChangeTimestampMonotonic=2381616958834
InactiveExitTimestamp=Thu 2022-04-21 14:55:31 CDT
InactiveExitTimestampMonotonic=2381616958834
ActiveEnterTimestamp=Thu 2022-04-21 14:55:31 CDT
ActiveEnterTimestampMonotonic=2381616958834
ActiveExitTimestamp=Thu 2022-04-21 14:55:30 CDT
ActiveExitTimestampMonotonic=2381615911078
InactiveEnterTimestamp=Thu 2022-04-21 14:55:31 CDT
InactiveEnterTimestampMonotonic=2381616953431
CanStart=yes
CanStop=yes
CanReload=no
CanIsolate=no
CanFreeze=yes
StopWhenUnneeded=no
RefuseManualStart=no
RefuseManualStop=no
AllowIsolate=no
DefaultDependencies=yes
OnFailureJobMode=replace
IgnoreOnIsolate=no
NeedDaemonReload=no
JobTimeoutUSec=infinity
JobRunningTimeoutUSec=infinity
JobTimeoutAction=none
ConditionResult=yes
AssertResult=yes
ConditionTimestamp=Thu 2022-04-21 14:55:31 CDT
ConditionTimestampMonotonic=2381616955804
AssertTimestamp=Thu 2022-04-21 14:55:31 CDT
AssertTimestampMonotonic=2381616955809
Transient=no
Perpetual=no
StartLimitIntervalUSec=10s
StartLimitBurst=5
StartLimitAction=none
FailureAction=none
SuccessAction=none
InvocationID=abe28dbf7bdf4b9ca7d5fd919cbf89c9
CollectMode=inactive

Database

SQLite

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions