Advanced
Mailing List Archive

Mailing List Archive

  1. Home >
  2. Varnish>
  3. Bugs
#1792: If few free space and file-storage on xfs, Varnish does not restart.
varnish-bugs at varnish-cache
Sep 14, 2015, 12:23 AM
Post #1 of 3 (1230 views)
Permalink
#1792: If few free space and file-storage on xfs, Varnish does not restart.
-------------------------+----------------------
Reporter: xcir | Type: defect
Status: new | Priority: normal
Milestone: | Component: varnishd
Version: 4.1.0-beta1 | Severity: normal
Keywords: |
-------------------------+----------------------
Hi, I got an error message that "Error: (-sfile) allocation error: No
space left on device" at the time of varnish restart.


Environment:
{{{
varnishd (varnish-4.1.0-beta1 revision 1628f0b)
Linux varnish-trunk 3.13.0-49-generic #83-Ubuntu SMP Fri Apr 10 20:11:33
UTC 2015 x86_64 x86_64 x86_64 GNU/Linux (Ubuntu 14.04.2 LTS)

Filesystem Type 1K-blocks Used Available Use% Mounted on
/dev/xvdh xfs 15718400 32928 15685472 1% /mnt/xfs
}}}

Log(xfs / Fail)
{{{
root@varnish-trunk:~/varnish-4.1.0-beta1# df -T /mnt/xfs
Filesystem Type 1K-blocks Used Available Use% Mounted on
/dev/xvdh xfs 15718400 32928 15685472 1% /mnt/xfs

root@varnish-trunk:~/varnish-4.1.0-beta1# /etc/init.d/varnish start
* Starting HTTP accelerator varnishd
...done.

root@varnish-trunk:~/varnish-4.1.0-beta1# df -T /mnt/xfs
Filesystem Type 1K-blocks Used Available Use% Mounted on
/dev/xvdh xfs 15718400 10518688 5199712 67% /mnt/xfs

root@varnish-trunk:~/varnish-4.1.0-beta1# du -sb
/mnt/xfs/varnish_storage.bin
10737418240 /mnt/xfs/varnish_storage.bin

root@varnish-trunk:~/varnish-4.1.0-beta1# ps axut|grep varn
varnish 14687 0.0 0.2 124768 5384 ? Ss 14:43 0:00
/usr/local/sbin/varnishd -P /run/varnishd.pid -a :6081 -T localhost:6082
-f /etc/varnish/default.vcl -S /etc/varnish/secret -p
thread_pool_stack=512k -s malloc,256m -s
xfs=file,/mnt/xfs/varnish_storage.bin,10G
varnish 14689 0.0 5.9 11074296 119456 ? Sl 14:43 0:00
/usr/local/sbin/varnishd -P /run/varnishd.pid -a :6081 -T localhost:6082
-f /etc/varnish/default.vcl -S /etc/varnish/secret -p
thread_pool_stack=512k -s malloc,256m -s
xfs=file,/mnt/xfs/varnish_storage.bin,10G
root 15168 0.0 0.0 8860 644 pts/1 R+ 14:49 0:00 grep
--color=auto varn

root@varnish-trunk:~/varnish-4.1.0-beta1# /etc/init.d/varnish restart
Error: (-sfile) allocation error: No space left on device
* Syntax check failed, not restarting
}}}

Log(ext4 / Success)
{{{

root@varnish-trunk:~/varnish-4.1.0-beta1# df -T /mnt/ext4
Filesystem Type 1K-blocks Used Available Use% Mounted on
/dev/xvdi ext4 15350768 38384 14509568 1% /mnt/ext4

root@varnish-trunk:~/varnish-4.1.0-beta1# /etc/init.d/varnish start
* Starting HTTP accelerator varnishd
...done.

root@varnish-trunk:~/varnish-4.1.0-beta1# df -T /mnt/ext4
Filesystem Type 1K-blocks Used Available Use% Mounted on
/dev/xvdi ext4 15350768 10524148 4023804 73% /mnt/ext4

root@varnish-trunk:~/varnish-4.1.0-beta1# du -sb
/mnt/xfs/varnish_storage.bin
10737418240 /mnt/xfs/varnish_storage.bin

root@varnish-trunk:~/varnish-4.1.0-beta1# ps axut|grep varn
varnish 15639 0.0 0.2 124768 5376 ? Ss 14:56 0:00
/usr/local/sbin/varnishd -P /run/varnishd.pid -a :6081 -T localhost:6082
-f /etc/varnish/default.vcl -S /etc/varnish/secret -p
thread_pool_stack=512k -s malloc,256m -s
ext4=file,/mnt/ext4/varnish_storage.bin,10G
varnish 15641 0.3 5.8 11074296 117260 ? Sl 14:56 0:00
/usr/local/sbin/varnishd -P /run/varnishd.pid -a :6081 -T localhost:6082
-f /etc/varnish/default.vcl -S /etc/varnish/secret -p
thread_pool_stack=512k -s malloc,256m -s
ext4=file,/mnt/ext4/varnish_storage.bin,10G
root 15904 0.0 0.0 8860 640 pts/1 S+ 14:56 0:00 grep
--color=auto varn

root@varnish-trunk:~/varnish-4.1.0-beta1# /etc/init.d/varnish restart
...

* Stopping HTTP accelerator varnishd
...done.
* Starting HTTP accelerator varnishd
...done.
}}}

The cause is to fail to fallocate in libvarnish/vfil.c (Even if
"(st.st_blocks * 512) + fsspace >= size")

https://github.com/varnish/Varnish-
Cache/blob/117c2bdcfbb8c11f48bd9137e76edfc746dfbe71/lib/libvarnish/vfil.c#L188

I think not require fallocate, if "st_blocks * 512" is greater than
st_size.
Because already allocated.

Log(patched)
{{{
root@varnish-trunk:~/varnish-4.1.0-beta1# df -T /mnt/xfs
Filesystem Type 1K-blocks Used Available Use% Mounted on
/dev/xvdh xfs 15718400 32928 15685472 1% /mnt/xfs
root@varnish-trunk:~/varnish-4.1.0-beta1# /etc/init.d/varnish start
* Starting HTTP accelerator varnishd
...done.
root@varnish-trunk:~/varnish-4.1.0-beta1# df -T /mnt/xfs
Filesystem Type 1K-blocks Used Available Use% Mounted on
/dev/xvdh xfs 15718400 10518688 5199712 67% /mnt/xfs


root@varnish-trunk:~/varnish-4.1.0-beta1# du -sb
/mnt/xfs/varnish_storage.bin
10737418240 /mnt/xfs/varnish_storage.bin

root@varnish-trunk:~/varnish-4.1.0-beta1# ps axut|grep varn
varnish 20804 0.0 0.2 124768 5380 ? Ss 16:20 0:00
/usr/local/sbin/varnishd -P /run/varnishd.pid -a :6081 -T localhost:6082
-f /etc/varnish/default.vcl -S /etc/varnish/secret -p
thread_pool_stack=512k -s malloc,256m -s
xfs=file,/mnt/xfs/varnish_storage.bin,10G
varnish 20806 0.1 5.9 11074296 119308 ? Sl 16:20 0:00
/usr/local/sbin/varnishd -P /run/varnishd.pid -a :6081 -T localhost:6082
-f /etc/varnish/default.vcl -S /etc/varnish/secret -p
thread_pool_stack=512k -s malloc,256m -s
xfs=file,/mnt/xfs/varnish_storage.bin,10G
root 21026 0.0 0.0 8860 648 pts/1 S+ 16:21 0:00 grep
--color=auto varn

root@varnish-trunk:~/varnish-4.1.0-beta1# /etc/init.d/varnish restart
...

* Stopping HTTP accelerator varnishd
...done.
* Starting HTTP accelerator varnishd
...done.

}}}

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1792>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1792: If few free space and file-storage on xfs, Varnish does not restart. [ In reply to ]
varnish-bugs at varnish-cache
Sep 14, 2015, 8:24 AM
Post #2 of 3 (1191 views)
Permalink
#1792: If few free space and file-storage on xfs, Varnish does not restart.
----------------------+--------------------------
Reporter: xcir | Owner:
Type: defect | Status: new
Priority: normal | Milestone:
Component: varnishd | Version: 4.1.0-beta1
Severity: normal | Resolution:
Keywords: |
----------------------+--------------------------

Comment (by xcir):

--Test code
{{{
root@varnish-trunk:~/varnish-4.1.0-beta1- diff -up
lib/libvarnish/vfil.c.org lib/libvarnish/vfil.c
--- lib/libvarnish/vfil.c.org 2015-09-14 23:59:00.228175000 +0900
+++ lib/libvarnish/vfil.c 2015-09-15 00:09:22.568175000 +0900
@@ -56,6 +56,7 @@
-include "vdef.h"
-include "vfil.h"

+-include <syslog.h>
char *
VFIL_readfd(int fd, ssize_t *sz)
{
@@ -182,9 +183,14 @@ VFIL_allocate(int fd, off_t size, int in
errno = ENOSPC;
return (-1);
}
+syslog(6,"VFIL:A:st_blocks*512:%ld st_size:%ld free-space:%ld require-
size:%ld errno:%d",st.st_blocks * 512, st.st_size, fsspace ,size,errno);
+// if(st.st_blocks * 512 >= st.st_size)
+// return (0);
-ifdef HAVE_FALLOCATE
if (!fallocate(fd, 0, 0, size))
return (0);
+syslog(6,"VFIL:B:st_blocks*512:%ld st_size:%ld free-space:%ld require-
size:%ld errno:%d",st.st_blocks * 512, st.st_size, fsspace ,size,errno);
+
if (errno == ENOSPC)
return (-1);
-endif
}}}

-xfs[[BR]]
--Start(1st)
{{{
Sep 15 00:04:00 varnish-trunk varnishd: VFIL:A:st_blocks*512:0
st_size:10737418240 free-space:16061923328 require-size:10737418240
errno:0
Sep 15 00:04:00 varnish-trunk varnishd[9463]: Platform:
Linux,3.13.0-49-generic,x86_64,-junix,-smalloc,-sfile,-smalloc,-hcritbit
Sep 15 00:04:00 varnish-trunk varnishd[9463]: VFIL:A:st_blocks*512:0
st_size:84934656 free-space:95130189824 require-size:84934656 errno:0
Sep 15 00:04:00 varnish-trunk varnishd[9463]: child (9465) Started
Sep 15 00:04:00 varnish-trunk varnishd[9463]: Child (9465) said Child
starts
Sep 15 00:04:00 varnish-trunk varnishd[9463]: Child (9465) said SMF.xfs
mmap'ed 10737418240 bytes of 10737418240
}}}

--Restart(Fail)
{{{
Sep 15 00:04:17 varnish-trunk varnishd: VFIL:A:st_blocks*512:10737418240
st_size:10737418240 free-space:5324505088 require-size:10737418240
errno:17
Sep 15 00:04:17 varnish-trunk varnishd: VFIL:B:st_blocks*512:10737418240
st_size:10737418240 free-space:5324505088 require-size:10737418240
errno:28
}}}

--Stop and Start(Fail)[[BR]]
---Stop
{{{
Sep 15 00:04:36 varnish-trunk varnishd[9463]: Manager got SIGINT
Sep 15 00:04:36 varnish-trunk varnishd[9463]: Stopping Child
Sep 15 00:04:37 varnish-trunk varnishd[9463]: Child (9465) ended
Sep 15 00:04:37 varnish-trunk varnishd[9463]: VFIL:A:st_blocks*512:0
st_size:84934656 free-space:95130189824 require-size:84934656 errno:0
Sep 15 00:04:37 varnish-trunk varnishd[9463]: Child (9465) said Child dies
Sep 15 00:04:37 varnish-trunk varnishd[9463]: Child cleanup complete
}}}
---Start(Fail)
{{{
Sep 15 00:04:40 varnish-trunk varnishd: VFIL:A:st_blocks*512:10737418240
st_size:10737418240 free-space:5324505088 require-size:10737418240
errno:17
Sep 15 00:04:40 varnish-trunk varnishd: VFIL:B:st_blocks*512:10737418240
st_size:10737418240 free-space:5324505088 require-size:10737418240
errno:28
}}}

-ext4[[BR]]
--Start(1st)
{{{
Sep 15 00:12:35 varnish-trunk varnishd: VFIL:A:st_blocks*512:0
st_size:10737418240 free-space:14857797632 require-size:10737418240
errno:0
Sep 15 00:12:35 varnish-trunk varnishd[10056]: Platform:
Linux,3.13.0-49-generic,x86_64,-junix,-smalloc,-sfile,-smalloc,-hcritbit
Sep 15 00:12:35 varnish-trunk varnishd[10056]: VFIL:A:st_blocks*512:0
st_size:84934656 free-space:95130181632 require-size:84934656 errno:0
Sep 15 00:12:35 varnish-trunk varnishd[10056]: child (10058) Started
Sep 15 00:12:35 varnish-trunk varnishd[10056]: Child (10058) said Child
starts
Sep 15 00:12:35 varnish-trunk varnishd[10056]: Child (10058) said SMF.ext4
mmap'ed 10737418240 bytes of 10737418240
}}}

--Restart(Success)
{{{
Sep 15 00:12:51 varnish-trunk varnishd: VFIL:A:st_blocks*512:10737422336
st_size:10737418240 free-space:4120375296 require-size:10737418240
errno:17
Sep 15 00:12:51 varnish-trunk varnishd[10056]: Manager got SIGINT
Sep 15 00:12:51 varnish-trunk varnishd[10056]: Stopping Child
Sep 15 00:12:52 varnish-trunk varnishd[10056]: Child (10058) ended
Sep 15 00:12:52 varnish-trunk varnishd[10056]: VFIL:A:st_blocks*512:0
st_size:84934656 free-space:95130173440 require-size:84934656 errno:0
Sep 15 00:12:52 varnish-trunk varnishd[10056]: Child (10058) said Child
dies
Sep 15 00:12:52 varnish-trunk varnishd[10056]: Child cleanup complete
Sep 15 00:12:52 varnish-trunk varnishd: VFIL:A:st_blocks*512:10737422336
st_size:10737418240 free-space:4120375296 require-size:10737418240
errno:17
Sep 15 00:12:52 varnish-trunk varnishd[10350]: Platform:
Linux,3.13.0-49-generic,x86_64,-junix,-smalloc,-sfile,-smalloc,-hcritbit
Sep 15 00:12:52 varnish-trunk varnishd[10350]: VFIL:A:st_blocks*512:0
st_size:84934656 free-space:95130173440 require-size:84934656 errno:0
Sep 15 00:12:52 varnish-trunk varnishd[10350]: child (10352) Started
Sep 15 00:12:52 varnish-trunk varnishd[10350]: Child (10352) said Child
starts
Sep 15 00:12:52 varnish-trunk varnishd[10350]: Child (10352) said SMF.ext4
mmap'ed 10737418240 bytes of 10737418240
}}}
--Stop and Start(Success)[[BR]]
---Stop
{{{
Sep 15 00:14:38 varnish-trunk varnishd[10350]: Manager got SIGINT
Sep 15 00:14:38 varnish-trunk varnishd[10350]: Stopping Child
Sep 15 00:14:39 varnish-trunk varnishd[10350]: Child (10352) ended
Sep 15 00:14:39 varnish-trunk varnishd[10350]: VFIL:A:st_blocks*512:0
st_size:84934656 free-space:95130181632 require-size:84934656 errno:0
Sep 15 00:14:39 varnish-trunk varnishd[10350]: Child (10352) said Child
dies
Sep 15 00:14:39 varnish-trunk varnishd[10350]: Child cleanup complete
}}}
---Start(Success)
{{{
Sep 15 00:14:52 varnish-trunk varnishd: VFIL:A:st_blocks*512:10737422336
st_size:10737418240 free-space:4120375296 require-size:10737418240
errno:17
Sep 15 00:14:52 varnish-trunk varnishd[10662]: Platform:
Linux,3.13.0-49-generic,x86_64,-junix,-smalloc,-sfile,-smalloc,-hcritbit
Sep 15 00:14:52 varnish-trunk varnishd[10662]: VFIL:A:st_blocks*512:0
st_size:84934656 free-space:95130181632 require-size:84934656 errno:0
Sep 15 00:14:52 varnish-trunk varnishd[10662]: child (10664) Started
Sep 15 00:14:52 varnish-trunk varnishd[10662]: Child (10664) said Child
starts
Sep 15 00:14:52 varnish-trunk varnishd[10662]: Child (10664) said SMF.ext4
mmap'ed 10737418240 bytes of 10737418240
}}}

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1792#comment:1>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1792: If few free space and file-storage on xfs, Varnish does not restart. [ In reply to ]
varnish-bugs at varnish-cache
Sep 15, 2015, 6:37 AM
Post #3 of 3 (1199 views)
Permalink
#1792: If few free space and file-storage on xfs, Varnish does not restart.
----------------------+-----------------------------------------------
Reporter: xcir | Owner: Martin Blix Grydeland <martin@…>
Type: defect | Status: closed
Priority: normal | Milestone:
Component: varnishd | Version: 4.1.0-beta1
Severity: normal | Resolution: fixed
Keywords: |
----------------------+-----------------------------------------------
Changes (by Martin Blix Grydeland <martin@…>):

* owner: => Martin Blix Grydeland <martin@…>
* status: new => closed
* resolution: => fixed


Comment:

In [9f38e1a8909baeb55f6d4e570948d3f904146303]:
{{{
#!CommitTicketReference repository=""
revision="9f38e1a8909baeb55f6d4e570948d3f904146303"
fallocate will for some filesystems (e.g. xfs) not take the already
allocated blocks of the file into account. This will cause fallocate
to report ENOSPC when called on an existing fully allocated file
unless the filesystem has enough free space to accomodate the complete
new file size. Because of this we enable fallocate only on filesystems
that are known to work as we expect.

Fixes: #1792
}}}

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1792#comment:2>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs

©2024 GT.net. A Gossamer Threads company.
5th Floor, 455 Granville St., Vancouver, BC V6C 1T1, Canada | Legal