Mailing List Archive

#1682: If file-storage pre-created and more than 50% of the file-system space, Varnish does not start.
#1682: If file-storage pre-created and more than 50% of the file-system space,
Varnish does not start.
-------------------+----------------------
Reporter: xcir | Type: defect
Status: new | Priority: normal
Milestone: | Component: varnishd
Version: 4.0.3 | Severity: normal
Keywords: |
-------------------+----------------------
Hi, I pre-create the storage file using the dd(1).

Unfortunately, I catch the "Error: (-sfile) size "XXG": larger than file
system" message at the varnish start.

And, If storage is full, Varnish can't restart.(specified size > file-
system space)

https://github.com/varnish/Varnish-
Cache/commit/551e0de9e89a83b6b465fc8800dfc9b47217a9dc

This commit does not care pre-allocation size.

I think that it is necessary to check the real size.

'''Storage is pre-created / Unpatched'''
{{{
root@varnish4:~# /var/opt/varnish-4.0.3/sbin/varnishd -V
varnishd (varnish-4.0.3 revision b8c4a34)
Copyright (c) 2006 Verdens Gang AS
Copyright (c) 2006-2014 Varnish Software AS

root@varnish4:~# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/xvda1 101195160 52082336 43972344 55% /
udev 975672 4 975668 1% /dev
tmpfs 199408 188 199220 1% /run
none 5120 0 5120 0% /run/lock
none 997024 0 997024 0% /run/shm

root@varnish4:~# /var/opt/varnish-4.0.3/sbin/varnishd -a :6083 -T
localhost:6084 -f /etc/varnish/default.vcl -S /etc/varnish/secret -p
feature=+esi_disable_xml_check -s
file,/var/lib/varnish/varnish_storage.bin,50G
Error: (-sfile) size "50G": larger than file system
}}}

'''Storage is pre-created / patched'''
{{{
root@varnish4:~# /var/opt/varnish-4branch/sbin/varnishd -V
varnishd (varnish-4.0.3 revision b8c4a34)
Copyright (c) 2006 Verdens Gang AS
Copyright (c) 2006-2014 Varnish Software AS

root@varnish4:~# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/xvda1 101195160 52082336 43972344 55% /
udev 975672 4 975668 1% /dev
tmpfs 199408 188 199220 1% /run
none 5120 0 5120 0% /run/lock
none 997024 0 997024 0% /run/shm

root@varnish4:~# /var/opt/varnish-4branch/sbin/varnishd -a :6083 -T
localhost:6084 -f /etc/varnish/default.vcl -S /etc/varnish/secret -p
feature=+esi_disable_xml_check -s
file,/var/lib/varnish/varnish_storage.bin,50G

root@varnish4:~# ps axut|grep varn
root 23799 15.5 4.2 114988 84232 ? SLs 23:28 0:00 /var/opt
/varnish-4branch/sbin/varnishd -a :6083 -T localhost:6084 -f
/etc/varnish/default.vcl -S /etc/varnish/secret -p
feature=+esi_disable_xml_check -s
file,/var/lib/varnish/varnish_storage.bin,50G
nobody 23801 4.0 4.2 53475620 84936 ? Sl 23:28 0:00 /var/opt
/varnish-4branch/sbin/varnishd -a :6083 -T localhost:6084 -f
/etc/varnish/default.vcl -S /etc/varnish/secret -p
feature=+esi_disable_xml_check -s
file,/var/lib/varnish/varnish_storage.bin,50G
root 24019 0.0 0.0 9388 932 pts/2 R+ 23:28 0:00 grep
--color=auto varn


}}}

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1682>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1682: If file-storage pre-created and more than 50% of the file-system space, Varnish does not start. [ In reply to ]
#1682: If file-storage pre-created and more than 50% of the file-system space,
Varnish does not start.
----------------------+--------------------
Reporter: xcir | Owner:
Type: defect | Status: new
Priority: normal | Milestone:
Component: varnishd | Version: 4.0.3
Severity: normal | Resolution:
Keywords: |
----------------------+--------------------

Comment (by martin):

This comes down to lack of backporting commits
22ad1c95f7a0a6588c1dc4d682183616587595ef and
1f4067c95f144d80cc619cca629d339b72635d92 to 4.0

Martin

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1682#comment:1>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1682: If file-storage pre-created and more than 50% of the file-system space, Varnish does not start. [ In reply to ]
#1682: If file-storage pre-created and more than 50% of the file-system space,
Varnish does not start.
----------------------+---------------------
Reporter: xcir | Owner:
Type: defect | Status: closed
Priority: normal | Milestone:
Component: varnishd | Version: 4.0.3
Severity: normal | Resolution: fixed
Keywords: |
----------------------+---------------------
Changes (by martin):

* status: new => closed
* resolution: => fixed


Comment:

These commits are in the 4.0 git branch. Closing ticket.

Martin

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1682#comment:2>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1682: If file-storage pre-created and more than 50% of the file-system space, Varnish does not start. [ In reply to ]
#1682: If file-storage pre-created and more than 50% of the file-system space,
Varnish does not start.
----------------------+-----------------------
Reporter: xcir | Owner:
Type: defect | Status: reopened
Priority: normal | Milestone:
Component: varnishd | Version: 4.0.3
Severity: normal | Resolution:
Keywords: |
----------------------+-----------------------
Changes (by xcir):

* status: closed => reopened
* resolution: fixed =>


Comment:

Hi, I got an error message that "Error: (-sfile) allocation error: No
space left on device" at the time of varnish restart.

Environment
{{{
varnishd (varnish-4.1.0-beta1 revision 017ec21) and varnishd (varnish-
trunk revision ca88bce)
Linux cache03 3.10.0-229.4.2.el7.x86_64 #1 SMP Wed May 13 10:06:09 UTC
2015 x86_64 x86_64 x86_64 GNU/Linux
CentOS Linux release 7.1.1503 (Core)
filesystem is xfs.
}}}
The cause is to fail to fallocate in libvarnish/vfil.c (Even if
"(st.st_blocks * 512) + fsspace >= size")

--fail log
{{{
loggin
syslog(6,"VFIL:st_blocks*512:%ld st_size:%ld free-space:%ld require-
size:%ld",st.st_blocks * 512, st.st_size, fsspace ,size);
syslog:
Sep 9 13:38:45 cache03 varnishd: VFIL:st_blocks*512:46170902528
st_size:46170898432 free-space:3188826112 require-size:46170898432
stdout:
Error: (-sfile) allocation error: No space left on device
}}}
https://github.com/varnish/Varnish-
Cache/blob/117c2bdcfbb8c11f48bd9137e76edfc746dfbe71/lib/libvarnish/vfil.c#L188

I think, if "st_blocks * 512" is greater than st_size is not require
fallocate. (Because already allocated)

I send patch.
https://www.varnish-cache.org/trac/attachment/ticket/1682/0001-If-
st_blocks-512-is-greater-than-st_size-is-not-requ.patch

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1682#comment:3>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1682: If file-storage pre-created and more than 50% of the file-system space, Varnish does not start. [ In reply to ]
#1682: If file-storage pre-created and more than 50% of the file-system space,
Varnish does not start.
----------------------+-----------------------
Reporter: xcir | Owner:
Type: defect | Status: reopened
Priority: normal | Milestone:
Component: varnishd | Version: 4.0.3
Severity: normal | Resolution:
Keywords: |
----------------------+-----------------------

Comment (by xcir):

I'm sorry. I misunderstood VFIL_allocate has been backported to 4.0.
I create new ticket for 4.1, if necessary.

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1682#comment:4>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1682: If file-storage pre-created and more than 50% of the file-system space, Varnish does not start. [ In reply to ]
#1682: If file-storage pre-created and more than 50% of the file-system space,
Varnish does not start.
----------------------+-----------------------
Reporter: xcir | Owner:
Type: defect | Status: reopened
Priority: normal | Milestone:
Component: varnishd | Version: 4.0.3
Severity: normal | Resolution:
Keywords: |
----------------------+-----------------------

Comment (by xcir):

I create another ticket.(#1792)
Could you close this ticket?

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1682#comment:5>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1682: If file-storage pre-created and more than 50% of the file-system space, Varnish does not start. [ In reply to ]
#1682: If file-storage pre-created and more than 50% of the file-system space,
Varnish does not start.
----------------------+------------------------
Reporter: xcir | Owner:
Type: defect | Status: closed
Priority: normal | Milestone:
Component: varnishd | Version: 4.0.3
Severity: normal | Resolution: duplicate
Keywords: |
----------------------+------------------------
Changes (by phk):

* status: reopened => closed
* resolution: => duplicate


--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1682#comment:6>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs