一、问题
之前已经写过一篇 "<<Linux磁盘占用率与使用率不符>>"
将进程kill掉就行
但是如果进程不能被停止呢
之前碰到一个例子:
朋友的系统盘空间爆满,du -sh查找不出问题,lsof|grep delete查出来是/tmp目录下一堆mysql进程的删除文件占着空间
ps了一下进程号,发现正好是mysql运行的进程,按照上面办法重启mysql肯定能解决问题,但是工作时间数据库跑着业务肯定没法重启
不重启又会因为系统盘爆满,mysql没指定临时目录(数据目录有单独磁盘,tmpdir没指定默认用/tmp目录),无法写到/tmp目录下,影响业务
网上搜了一下临时解决办法,通过清空文件释放空间。
处理办法如下:
通过lsof | grep deleted 找到未能删除掉的文件,确定占用的进程号;
通过 ls -l /proc/PID/fd/* | grep 文件名,找到相应文件句柄;
"比如:lr-x------ 1 root root 64 May 23 16:05 3 -> /delete.tmp (deleted)"
清除文件内容 echo > /proc/PID/fd/FD_NUM #此处FD_NUM为上一步查出来的文件句柄,通常为一个数字
这个操作被不会将文件删除,而是通过将文档内容清空的方法释放空间,文件还是存在的。
如果此文件还在往里面写数据,还是会重新膨胀起来占据空间
这只是一个临时解决办法,最终还是要通过重启释放
二、附录/proc/PID/fd
那么/proc/PID/fd 是啥呢?标签:deleted,lsof,dev,standard,inode,fd,file,Linux,proc From: https://blog.51cto.com/u_13236892/5787402
man proc
/proc/[pid]/fd/
This is a subdirectory containing one entry for each file
which the process has open, named by its file descriptor,
and which is a symbolic link to the actual file. Thus, 0
is standard input, 1 standard output, 2 standard error,
and so on.
For file descriptors for pipes and sockets, the entries
will be symbolic links whose content is the file type with
the inode. A readlink(2) call on this file returns a
string in the format:
type:[inode]
For example, socket:[2248868] will be a socket and its
inode is 2248868. For sockets, that inode can be used to
find more information in one of the files under
/proc/net/.
For file descriptors that have no corresponding inode
(e.g., file descriptors produced by bpf(2),
epoll_create(2), eventfd(2), inotify_init(2),
perf_event_open(2), signalfd(2), timerfd_create(2), and
userfaultfd(2)), the entry will be a symbolic link with
contents of the form
anon_inode:<file-type>
In many cases (but not all), the file-type is surrounded
by square brackets.
For example, an epoll file descriptor will have a symbolic
link whose content is the string anon_inode:[eventpoll].
In a multithreaded process, the contents of this directory
are not available if the main thread has already
terminated (typically by calling pthread_exit(3)).
Programs that take a filename as a command-line argument,
but don't take input from standard input if no argument is
supplied, and programs that write to a file named as a
command-line argument, but don't send their output to
standard output if no argument is supplied, can
nevertheless be made to use standard input or standard
output by using /proc/[pid]/fd files as command-line
arguments. For example, assuming that -i is the flag
designating an input file and -o is the flag designating
an output file:
$ foobar -i /proc/self/fd/0 -o /proc/self/fd/1 ...
and you have a working filter.
/proc/self/fd/N is approximately the same as /dev/fd/N in
some UNIX and UNIX-like systems. Most Linux MAKEDEV
scripts symbolically link /dev/fd to /proc/self/fd, in
fact.
Most systems provide symbolic links /dev/stdin,
/dev/stdout, and /dev/stderr, which respectively link to
the files 0, 1, and 2 in /proc/self/fd. Thus the example
command above could be written as:
$ foobar -i /dev/stdin -o /dev/stdout ...
Permission to dereference or read (readlink(2)) the
symbolic links in this directory is governed by a ptrace
access mode PTRACE_MODE_READ_FSCREDS check; see ptrace(2).
Note that for file descriptors referring to inodes (pipes
and sockets, see above), those inodes still have
permission bits and ownership information distinct from
those of the /proc/[pid]/fd entry, and that the owner may
differ from the user and group IDs of the process. An
unprivileged process may lack permissions to open them, as
in this example:
$ echo test | sudo -u nobody cat
test
$ echo test | sudo -u nobody cat /proc/self/fd/0
cat: /proc/self/fd/0: Permission denied
File descriptor 0 refers to the pipe created by the shell
and owned by that shell's user, which is not nobody, so
cat does not have permission to create a new file
descriptor to read from that inode, even though it can
still read from its existing file descriptor 0.