Mailing List Archive: database is full

database is full

Aug 17, 2004, 7:31 AM

Post #1 of 6 (1220 views)

I've been running trac for our in-house development for several months
now, and everything has run smoothly. Tonight, however, every operation
which writes to the database raises a 'database is full' exception.
Here's a sample traceback, produced when I try to save my edit to a wiki
page:

-----------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.3/site-packages/trac/core.py", line 475, in cgi_start
real_cgi_start()
File "/usr/lib/python2.3/site-packages/trac/core.py", line 470, in real_cgi_start
dispatch_request(path_info, args, req, env, database)
File "/usr/lib/python2.3/site-packages/trac/core.py", line 380, in dispatch_request
module.run()
File "/usr/lib/python2.3/site-packages/trac/Module.py", line 41, in run
self.render()
File "/usr/lib/python2.3/site-packages/trac/Wiki.py", line 758, in render
self.page.commit(author, comment, self.req.remote_addr)
File "/usr/lib/python2.3/site-packages/trac/Wiki.py", line 585, in commit
author, remote_addr, self.text, comment)
File "/usr/lib/python2.3/site-packages/sqlite/main.py", line 237, in execute
self.con._begin()
File "/usr/lib/python2.3/site-packages/sqlite/main.py", line 503, in _begin
self.db.execute("BEGIN")
DatabaseError: database is full
-----------------------------------------------

The disk isn't full, and permissions are set correctly. In fact, I have
no idea what changed between yesterday and today. So I started hunting
around a bit. I can create a new trac project with trac-admin, and the
database gets filled up with all the revision information as it should.
Same exception when I modify anything through the web interface, though.

To really narrow it down, I pulled up python and manually ran the
statements which produced the wiki error on the same db:

>>> import sqlite
>>> c = sqlite.connect('trac.db')
>>> x = c.cursor()
>>> x.execute('INSERT INTO WIKI (name, version, time, author, ipnr,
text, comment) VALUES (%s, %s, %s, %s, %s, %s, %s)', 'foo', 1, 2, 'cap',
'1.2.3.4', 'heres the stupid wiki page', 'and a comment even')
>>> c.commit()

Lo, the new page is now in my index, and I can view it, but changes are
verboten. I've rebooted the server and restarted apache several times. I
don't know enough about SQLite or Trac to have any clue what could be
going on. Help?

Version Info:
Debian/testing
Trac 0.7.1
Apache 2.0.47-1 (.deb)
SQLite 2.8.15-2 (.deb, Upgraded from 2.8.13-3, which I was running when
this started happening)
PySQLite 0.5.1 (Upgraded from 0.5.0, ditto)

database is full [ In reply to ]

veliashberg at yahoo

Aug 17, 2004, 4:54 PM

Post #2 of 6 (1204 views)

Permalink

Just a thought - SQLite uses a single file to manage the entire database.
Could it be hitting the OS limit for the file (2G for linux, usually)? What
is the size of your trac.db file?

Also, Trac is fairly inefficient in how it stores Wiki updates (it stores
entire copy of a new version, not just diffs) - do you have lots of Wiki
pages that are actively updated?

Cheers,

Alik

--- Cap Petschulat <cap@cdres.com> wrote:

> I've been running trac for our in-house development for several months
> now, and everything has run smoothly. Tonight, however, every operation
> which writes to the database raises a 'database is full' exception.
> Here's a sample traceback, produced when I try to save my edit to a wiki
> page:
>
> -----------------------------------------------
> Traceback (most recent call last):
> File "/usr/lib/python2.3/site-packages/trac/core.py", line 475, in
> cgi_start
> real_cgi_start()
> File "/usr/lib/python2.3/site-packages/trac/core.py", line 470, in
> real_cgi_start
> dispatch_request(path_info, args, req, env, database)
> File "/usr/lib/python2.3/site-packages/trac/core.py", line 380, in
> dispatch_request
> module.run()
> File "/usr/lib/python2.3/site-packages/trac/Module.py", line 41, in run
> self.render()
> File "/usr/lib/python2.3/site-packages/trac/Wiki.py", line 758, in render
> self.page.commit(author, comment, self.req.remote_addr)
> File "/usr/lib/python2.3/site-packages/trac/Wiki.py", line 585, in commit
> author, remote_addr, self.text, comment)
> File "/usr/lib/python2.3/site-packages/sqlite/main.py", line 237, in
> execute
> self.con._begin()
> File "/usr/lib/python2.3/site-packages/sqlite/main.py", line 503, in
> _begin
> self.db.execute("BEGIN")
> DatabaseError: database is full
> -----------------------------------------------
>
> The disk isn't full, and permissions are set correctly. In fact, I have
> no idea what changed between yesterday and today. So I started hunting
> around a bit. I can create a new trac project with trac-admin, and the
> database gets filled up with all the revision information as it should.
> Same exception when I modify anything through the web interface, though.
>
> To really narrow it down, I pulled up python and manually ran the
> statements which produced the wiki error on the same db:
>
> >>> import sqlite
> >>> c = sqlite.connect('trac.db')
> >>> x = c.cursor()
> >>> x.execute('INSERT INTO WIKI (name, version, time, author, ipnr,
> text, comment) VALUES (%s, %s, %s, %s, %s, %s, %s)', 'foo', 1, 2, 'cap',
> '1.2.3.4', 'heres the stupid wiki page', 'and a comment even')
> >>> c.commit()
>
> Lo, the new page is now in my index, and I can view it, but changes are
> verboten. I've rebooted the server and restarted apache several times. I
> don't know enough about SQLite or Trac to have any clue what could be
> going on. Help?
>
> Version Info:
> Debian/testing
> Trac 0.7.1
> Apache 2.0.47-1 (.deb)
> SQLite 2.8.15-2 (.deb, Upgraded from 2.8.13-3, which I was running when
> this started happening)
> PySQLite 0.5.1 (Upgraded from 0.5.0, ditto)
>
> _______________________________________________
> Trac mailing list
> Trac@lists.edgewall.com
> http://lists.edgewall.com/mailman/listinfo/trac
>

__________________________________
Do you Yahoo!?
Yahoo! Mail - 50x more storage than other providers!
http://promotions.yahoo.com/new_mail

database is full [ In reply to ]

lists at cyberlot

Aug 17, 2004, 5:02 PM

Post #3 of 6 (1198 views)

Permalink

What would be nice is if trac had a "Wikie Prune" option that allowed
you to easily prune versions older then X number of days.

I think this would solve some of the problems and comments on tracs
"inefficient" wiki storage.

Alik Eliashberg wrote:
> Just a thought - SQLite uses a single file to manage the entire database.
> Could it be hitting the OS limit for the file (2G for linux, usually)? What
> is the size of your trac.db file?
>
> Also, Trac is fairly inefficient in how it stores Wiki updates (it stores
> entire copy of a new version, not just diffs) - do you have lots of Wiki
> pages that are actively updated?
>
> Cheers,
>
> Alik
>
> --- Cap Petschulat <cap@cdres.com> wrote:
>
>
>>I've been running trac for our in-house development for several months
>>now, and everything has run smoothly. Tonight, however, every operation
>>which writes to the database raises a 'database is full' exception.
>>Here's a sample traceback, produced when I try to save my edit to a wiki
>>page:
>>
>>-----------------------------------------------
>>Traceback (most recent call last):
>> File "/usr/lib/python2.3/site-packages/trac/core.py", line 475, in
>>cgi_start
>> real_cgi_start()
>> File "/usr/lib/python2.3/site-packages/trac/core.py", line 470, in
>>real_cgi_start
>> dispatch_request(path_info, args, req, env, database)
>> File "/usr/lib/python2.3/site-packages/trac/core.py", line 380, in
>>dispatch_request
>> module.run()
>> File "/usr/lib/python2.3/site-packages/trac/Module.py", line 41, in run
>> self.render()
>> File "/usr/lib/python2.3/site-packages/trac/Wiki.py", line 758, in render
>> self.page.commit(author, comment, self.req.remote_addr)
>> File "/usr/lib/python2.3/site-packages/trac/Wiki.py", line 585, in commit
>> author, remote_addr, self.text, comment)
>> File "/usr/lib/python2.3/site-packages/sqlite/main.py", line 237, in
>>execute
>> self.con._begin()
>> File "/usr/lib/python2.3/site-packages/sqlite/main.py", line 503, in
>>_begin
>> self.db.execute("BEGIN")
>>DatabaseError: database is full
>>-----------------------------------------------
>>
>>The disk isn't full, and permissions are set correctly. In fact, I have
>>no idea what changed between yesterday and today. So I started hunting
>>around a bit. I can create a new trac project with trac-admin, and the
>>database gets filled up with all the revision information as it should.
>>Same exception when I modify anything through the web interface, though.
>>
>>To really narrow it down, I pulled up python and manually ran the
>>statements which produced the wiki error on the same db:
>>
>>
>>>>>import sqlite
>>>>>c = sqlite.connect('trac.db')
>>>>>x = c.cursor()
>>>>>x.execute('INSERT INTO WIKI (name, version, time, author, ipnr,
>>
>>text, comment) VALUES (%s, %s, %s, %s, %s, %s, %s)', 'foo', 1, 2, 'cap',
>>'1.2.3.4', 'heres the stupid wiki page', 'and a comment even')
>>
>>>>>c.commit()
>>
>>Lo, the new page is now in my index, and I can view it, but changes are
>>verboten. I've rebooted the server and restarted apache several times. I
>>don't know enough about SQLite or Trac to have any clue what could be
>>going on. Help?
>>
>>Version Info:
>>Debian/testing
>>Trac 0.7.1
>>Apache 2.0.47-1 (.deb)
>>SQLite 2.8.15-2 (.deb, Upgraded from 2.8.13-3, which I was running when
>>this started happening)
>>PySQLite 0.5.1 (Upgraded from 0.5.0, ditto)
>>
>>_______________________________________________
>>Trac mailing list
>>Trac@lists.edgewall.com
>>http://lists.edgewall.com/mailman/listinfo/trac
>>
>
>
>
>
>
> __________________________________
> Do you Yahoo!?
> Yahoo! Mail - 50x more storage than other providers!
> http://promotions.yahoo.com/new_mail
> _______________________________________________
> Trac mailing list
> Trac@lists.edgewall.com
> http://lists.edgewall.com/mailman/listinfo/trac
>

--
Richard Thomas
Cyberlot Technologies Group Inc.
507.398.4124 - Voice

database is full [ In reply to ]

veliashberg at yahoo

Aug 17, 2004, 5:18 PM

Post #4 of 6 (1194 views)

Permalink

--- Richard Thomas <lists@cyberlot.net> wrote:
> What would be nice is if trac had a "Wikie Prune" option that allowed
> you to easily prune versions older then X number of days.
>
> I think this would solve some of the problems and comments on tracs
> "inefficient" wiki storage.
I am not sure I agree with that. Having history is very valuable and given
the price of diskspace these days, there is no reason not to keep it.
I do think that with a single file limit on the database, it is not well
suited to store actual content (e.g. wiki). Per Jonas, Trac relies heavily on
SQL backend, so removing Wiki out of the DB is not feasible. Maybe the right
solution would be to store diffs (as opposed to a copy of the entire page)?

Alternatively it would be nice if Trac supported an arbitrary SQL-92
compliant backend (e.g. Postgress, MySQL, Oracle, etc.). This definetely has
the cost in administration (since the database would have to be
tuned/backed-up/managed) - but for large projects, chances are that there is
already a database that is being used for something else, so the incremental
cost would be negligible (especially, if Trac trully stays db-independent, as
opposed to require, say MySQL). I would guess, though, that switching to
database-agnostic architecture is well beyond the scope of 1.0 release.

Cheers,

Alik

>
> Alik Eliashberg wrote:
> > Just a thought - SQLite uses a single file to manage the entire database.
> > Could it be hitting the OS limit for the file (2G for linux, usually)?
> What
> > is the size of your trac.db file?
> >
> > Also, Trac is fairly inefficient in how it stores Wiki updates (it stores
> > entire copy of a new version, not just diffs) - do you have lots of Wiki
> > pages that are actively updated?
> >
> > Cheers,
> >
> > Alik
> >
> > --- Cap Petschulat <cap@cdres.com> wrote:
> >
> >
> >>I've been running trac for our in-house development for several months
> >>now, and everything has run smoothly. Tonight, however, every operation
> >>which writes to the database raises a 'database is full' exception.
> >>Here's a sample traceback, produced when I try to save my edit to a wiki
> >>page:
> >>
> >>-----------------------------------------------
> >>Traceback (most recent call last):
> >> File "/usr/lib/python2.3/site-packages/trac/core.py", line 475, in
> >>cgi_start
> >> real_cgi_start()
> >> File "/usr/lib/python2.3/site-packages/trac/core.py", line 470, in
> >>real_cgi_start
> >> dispatch_request(path_info, args, req, env, database)
> >> File "/usr/lib/python2.3/site-packages/trac/core.py", line 380, in
> >>dispatch_request
> >> module.run()
> >> File "/usr/lib/python2.3/site-packages/trac/Module.py", line 41, in run
> >> self.render()
> >> File "/usr/lib/python2.3/site-packages/trac/Wiki.py", line 758, in
> render
> >> self.page.commit(author, comment, self.req.remote_addr)
> >> File "/usr/lib/python2.3/site-packages/trac/Wiki.py", line 585, in
> commit
> >> author, remote_addr, self.text, comment)
> >> File "/usr/lib/python2.3/site-packages/sqlite/main.py", line 237, in
> >>execute
> >> self.con._begin()
> >> File "/usr/lib/python2.3/site-packages/sqlite/main.py", line 503, in
> >>_begin
> >> self.db.execute("BEGIN")
> >>DatabaseError: database is full
> >>-----------------------------------------------
> >>
> >>The disk isn't full, and permissions are set correctly. In fact, I have
> >>no idea what changed between yesterday and today. So I started hunting
> >>around a bit. I can create a new trac project with trac-admin, and the
> >>database gets filled up with all the revision information as it should.
> >>Same exception when I modify anything through the web interface, though.
> >>
> >>To really narrow it down, I pulled up python and manually ran the
> >>statements which produced the wiki error on the same db:
> >>
> >>
> >>>>>import sqlite
> >>>>>c = sqlite.connect('trac.db')
> >>>>>x = c.cursor()
> >>>>>x.execute('INSERT INTO WIKI (name, version, time, author, ipnr,
> >>
> >>text, comment) VALUES (%s, %s, %s, %s, %s, %s, %s)', 'foo', 1, 2, 'cap',
> >>'1.2.3.4', 'heres the stupid wiki page', 'and a comment even')
> >>
> >>>>>c.commit()
> >>
> >>Lo, the new page is now in my index, and I can view it, but changes are
> >>verboten. I've rebooted the server and restarted apache several times. I
> >>don't know enough about SQLite or Trac to have any clue what could be
> >>going on. Help?
> >>
> >>Version Info:
> >>Debian/testing
> >>Trac 0.7.1
> >>Apache 2.0.47-1 (.deb)
> >>SQLite 2.8.15-2 (.deb, Upgraded from 2.8.13-3, which I was running when
> >>this started happening)
> >>PySQLite 0.5.1 (Upgraded from 0.5.0, ditto)
> >>
> >>_______________________________________________
> >>Trac mailing list
> >>Trac@lists.edgewall.com
> >>http://lists.edgewall.com/mailman/listinfo/trac
> >>
> >
> >
> >
> >
> >
> > __________________________________
> > Do you Yahoo!?
> > Yahoo! Mail - 50x more storage than other providers!
> > http://promotions.yahoo.com/new_mail
> > _______________________________________________
> > Trac mailing list
> > Trac@lists.edgewall.com
> > http://lists.edgewall.com/mailman/listinfo/trac
> >
>
>
> --
> Richard Thomas
> Cyberlot Technologies Group Inc.
> 507.398.4124 - Voice
> _______________________________________________
> Trac mailing list
> Trac@lists.edgewall.com
> http://lists.edgewall.com/mailman/listinfo/trac
>

__________________________________
Do you Yahoo!?
Yahoo! Mail - You care about security. So do we.
http://promotions.yahoo.com/new_mail

database is full [ In reply to ]

veliashberg at yahoo

Aug 17, 2004, 5:29 PM

Post #5 of 6 (1203 views)

Permalink

My appologies for spreading the FUD. I checked SQLite's site and it says
(http://www.sqlite.org/faq.html#q10):

> As of version 2.7.4, SQLite can handle databases up to 2^41 bytes (2
> terabytes) in size on both Windows and Unix. Older version of SQLite
> were limited to databases of 2^31 bytes (2 gigabytes).

This means that as long as your FS can support the 2TB file, you should be
OK. I know that on Linux it is definetely possibly (but might require
recompiling the kernel and possibly playing with block size on Ext2/3)

If that is the case (and if there is no significant performance penalty),
then there trully is no reason to worry about Wiki size.

Cheers,

Alik

--- Alik Eliashberg <veliashberg@yahoo.com> wrote:

> Just a thought - SQLite uses a single file to manage the entire database.
> Could it be hitting the OS limit for the file (2G for linux, usually)? What
> is the size of your trac.db file?
>
> Also, Trac is fairly inefficient in how it stores Wiki updates (it stores
> entire copy of a new version, not just diffs) - do you have lots of Wiki
> pages that are actively updated?
>
> Cheers,
>
> Alik
>
> --- Cap Petschulat <cap@cdres.com> wrote:
>
> > I've been running trac for our in-house development for several months
> > now, and everything has run smoothly. Tonight, however, every operation
> > which writes to the database raises a 'database is full' exception.
> > Here's a sample traceback, produced when I try to save my edit to a wiki
> > page:
> >
> > -----------------------------------------------
> > Traceback (most recent call last):
> > File "/usr/lib/python2.3/site-packages/trac/core.py", line 475, in
> > cgi_start
> > real_cgi_start()
> > File "/usr/lib/python2.3/site-packages/trac/core.py", line 470, in
> > real_cgi_start
> > dispatch_request(path_info, args, req, env, database)
> > File "/usr/lib/python2.3/site-packages/trac/core.py", line 380, in
> > dispatch_request
> > module.run()
> > File "/usr/lib/python2.3/site-packages/trac/Module.py", line 41, in run
> > self.render()
> > File "/usr/lib/python2.3/site-packages/trac/Wiki.py", line 758, in
> render
> > self.page.commit(author, comment, self.req.remote_addr)
> > File "/usr/lib/python2.3/site-packages/trac/Wiki.py", line 585, in
> commit
> > author, remote_addr, self.text, comment)
> > File "/usr/lib/python2.3/site-packages/sqlite/main.py", line 237, in
> > execute
> > self.con._begin()
> > File "/usr/lib/python2.3/site-packages/sqlite/main.py", line 503, in
> > _begin
> > self.db.execute("BEGIN")
> > DatabaseError: database is full
> > -----------------------------------------------
> >
> > The disk isn't full, and permissions are set correctly. In fact, I have
> > no idea what changed between yesterday and today. So I started hunting
> > around a bit. I can create a new trac project with trac-admin, and the
> > database gets filled up with all the revision information as it should.
> > Same exception when I modify anything through the web interface, though.
> >
> > To really narrow it down, I pulled up python and manually ran the
> > statements which produced the wiki error on the same db:
> >
> > >>> import sqlite
> > >>> c = sqlite.connect('trac.db')
> > >>> x = c.cursor()
> > >>> x.execute('INSERT INTO WIKI (name, version, time, author, ipnr,
> > text, comment) VALUES (%s, %s, %s, %s, %s, %s, %s)', 'foo', 1, 2, 'cap',
> > '1.2.3.4', 'heres the stupid wiki page', 'and a comment even')
> > >>> c.commit()
> >
> > Lo, the new page is now in my index, and I can view it, but changes are
> > verboten. I've rebooted the server and restarted apache several times. I
> > don't know enough about SQLite or Trac to have any clue what could be
> > going on. Help?
> >
> > Version Info:
> > Debian/testing
> > Trac 0.7.1
> > Apache 2.0.47-1 (.deb)
> > SQLite 2.8.15-2 (.deb, Upgraded from 2.8.13-3, which I was running when
> > this started happening)
> > PySQLite 0.5.1 (Upgraded from 0.5.0, ditto)
> >
> > _______________________________________________
> > Trac mailing list
> > Trac@lists.edgewall.com
> > http://lists.edgewall.com/mailman/listinfo/trac
> >
>
>
>
>
> __________________________________
> Do you Yahoo!?
> Yahoo! Mail - 50x more storage than other providers!
> http://promotions.yahoo.com/new_mail
> _______________________________________________
> Trac mailing list
> Trac@lists.edgewall.com
> http://lists.edgewall.com/mailman/listinfo/trac
>

__________________________________
Do you Yahoo!?
Yahoo! Mail is new and improved - Check it out!
http://promotions.yahoo.com/new_mail

database is full [ In reply to ]

joel at rosdahl

Aug 17, 2004, 6:17 PM

Post #6 of 6 (1209 views)

Permalink

Cap Petschulat <cap@cdres.com> writes:

> [...]
> DatabaseError: database is full
> -----------------------------------------------
>
> The disk isn't full, and permissions are set correctly.
> [...]

You say that the disk (where the database file resides, I presume)
isn't full, but you should also check the partition where /tmp is
mounted since SQLite needs to be able to write there too.

Joel

--
Joel Rosdahl <joel@rosdahl.net>
Key BB845E97; fingerprint 9F4B D780 6EF4 5700 778D 8B22 0064 F9FF BB84 5E97