Mailing List Archive

IC freezes
Hi!

Is there a way to debug why ic get's stuck for up to 1-2 minutes when
updating eg the locale-table via the AdminUI?

René

_______________________________________________
interchange-users mailing list
interchange-users@icdevgroup.org
http://www.icdevgroup.org/mailman/listinfo/interchange-users
Re: IC freezes [ In reply to ]
> Hi!
>
> Is there a way to debug why ic get's stuck for up to 1-2 minutes when updating eg the locale-table via the AdminUI?
>
> René

I haven’t heard of this issue before; anything in the logs?

--
David Christensen
End Point Corporation
david@endpoint.com
785-727-1171
Re: IC freezes [ In reply to ]
Quoting David Christensen (david@endpoint.com):
> > Hi!
> >
> > Is there a way to debug why ic get's stuck for up to 1-2 minutes when updating eg the locale-table via the AdminUI?
> >
> > Ren??
>
> I haven???t heard of this issue before; anything in the logs?
>

If you have your locale database in GDBM, it could lock out the admin
while it was being updated.

--
Find the grain of truth in criticism, chew it, and swallow
it. -- anonymous

_______________________________________________
interchange-users mailing list
interchange-users@icdevgroup.org
http://www.icdevgroup.org/mailman/listinfo/interchange-users
Re: IC freezes [ In reply to ]
On 15.06.2018 15:22, M Heins wrote:
> Quoting David Christensen (david@endpoint.com):
>>> Hi!
>>>
>>> Is there a way to debug why ic get's stuck for up to 1-2 minutes when updating eg the locale-table via the AdminUI?
>>>
>>> Ren??
>>
>> I haven???t heard of this issue before; anything in the logs?
>>
>
> If you have your locale database in GDBM, it could lock out the admin
> while it was being updated.
>

Hi!

I can not find anything in the logs. What came into my mind is that
maybe my perl + modules have wrong versions? I run debian Stretch (i386)
and system Perl, so maybe this might be a problem?

I also tried to find some discussions on how to move the locale-database
away from GDBM, but i did not find such discussion, even that if
remember that this was discussed many years ago.. :-( Isn't the idea by
having the locale in GDBM that this part is in the memory, and any extra
db-lookups will not spoil the speed?

Here is btw info what the admin-ui is giving me about modules and db..
Maybe someone can spot some problematic module-versions?

Perl 5.24.1 (called with: /usr/bin/perl)
Optional Module Information

Archive::Tar found (v2.30)
LWP::Simple found (v6.34)
Archive::Zip found (v1.59)
MIME::Base64 found (v3.15)
Business::UPS not found.
Enables lookup of shipping costs directly from www.ups.com.
Safe::Hole found (v0.14)
Compress::Zlib not found.
Only needed for supplementary UserTag definitions.
Set::Crontab found (v1.03)
Crypt::Random found (v1.25)
Spreadsheet::ParseExcel found (v0.65)
Crypt::SSLeay found (v0.7304)
Spreadsheet::WriteExcel found (v2.40)
DBI found (v1.641)
Storable found (v3.11)
Digest::Bcrypt found (v1.209)
Tie::ShadowHash found (v1.00)
Digest::MD5 found (v2.55)
Tie::Watch not found.
Minor: cannot set watch points in catalog.cfg.
Digest::SHA found (v6.02)
URI::URL found (v5.04)
Image::Size found (v3.3)


GDBM available (v1.15)
No Berkeley DB_File.
LDAP available (v0.65)
DBI enabled (v1.641), available drivers:

DBM
ExampleP
File
Gofer
Mem
Pg
Proxy
SQLite
Sponge
mysql


Best Regards,

René

_______________________________________________
interchange-users mailing list
interchange-users@icdevgroup.org
http://www.icdevgroup.org/mailman/listinfo/interchange-users
Re: IC freezes [ In reply to ]
On 20.06.2018 14:33, René wrote:
> On 15.06.2018 15:22, M Heins wrote:
>> Quoting David Christensen (david@endpoint.com):
>>>> Hi!
>>>>
>>>> Is there a way to debug why ic get's stuck for up to 1-2 minutes
>>>> when updating eg the locale-table via the AdminUI?
>>>>
>>>> Ren??
>>>
>>> I haven???t heard of this issue before; anything in the logs?
>>>
>>
>> If you have your locale database in GDBM, it could lock out the admin
>> while it was being updated.
>>
>
> Hi!
>
> I can not find anything in the logs. What came into my mind is that
> maybe my perl + modules have wrong versions? I run debian Stretch (i386)
> and system Perl, so maybe this might be a problem?
>
> I also tried to find some discussions on how to move the locale-database
> away from GDBM, but i did not find such discussion, even that if
> remember that this was discussed many years ago.. :-( Isn't the idea by
> having the locale in GDBM that this part is in the memory, and any extra
> db-lookups will not spoil the speed?
>
> Here is btw info what the admin-ui is giving me about modules and db..
> Maybe someone can spot some problematic module-versions?
>
> Perl 5.24.1 (called with: /usr/bin/perl)
> Optional Module Information
>
> Archive::Tar found (v2.30)
> LWP::Simple found (v6.34)
> Archive::Zip found (v1.59)
> MIME::Base64 found (v3.15)
> Business::UPS not found.
> Enables lookup of shipping costs directly from www.ups.com.
> Safe::Hole found (v0.14)
> Compress::Zlib not found.
> Only needed for supplementary UserTag definitions.
> Set::Crontab found (v1.03)
> Crypt::Random found (v1.25)
> Spreadsheet::ParseExcel found (v0.65)
> Crypt::SSLeay found (v0.7304)
> Spreadsheet::WriteExcel found (v2.40)
> DBI found (v1.641)
> Storable found (v3.11)
> Digest::Bcrypt found (v1.209)
> Tie::ShadowHash found (v1.00)
> Digest::MD5 found (v2.55)
> Tie::Watch not found.
> Minor: cannot set watch points in catalog.cfg.
> Digest::SHA found (v6.02)
> URI::URL found (v5.04)
> Image::Size found (v3.3)
>
>
> GDBM available (v1.15)
> No Berkeley DB_File.
> LDAP available (v0.65)
> DBI enabled (v1.641), available drivers:
>
>     DBM
>     ExampleP
>     File
>     Gofer
>     Mem
>     Pg
>     Proxy
>     SQLite
>     Sponge
>     mysql

I'm replying to my self. I just installed a new catalog under the same
ic-instance, and there is still something strange. If i edit the
locale-table via the admin-ui, all catalogs "freeze" for a certain
period before the "lock/timeout" seems to get released and all pages
work again smoothly.

I'll try now with a fresh perlbrew-version of 5.24.1 and a fresh
ic-install to see if i still have the same problem..

René

_______________________________________________
interchange-users mailing list
interchange-users@icdevgroup.org
http://www.icdevgroup.org/mailman/listinfo/interchange-users
Re: IC freezes [ In reply to ]
>
> I'm replying to my self. I just installed a new catalog under the same ic-instance, and there is still something strange. If i edit the locale-table via the admin-ui, all catalogs "freeze" for a certain period before the "lock/timeout" seems to get released and all pages work again smoothly.
>
> I'll try now with a fresh perlbrew-version of 5.24.1 and a fresh ic-install to see if i still have the same problem..

When this is locked up is it responding to other requests? (You’d want to check in a different browser, as it locks sessions from the same session.) Basically trying to see if this is a local session-locking issue, or if it is a global IC lockup issue.

Best,

David
--
David Christensen
End Point Corporation
david@endpoint.com
785-727-1171
Re: IC freezes [ In reply to ]
On 25.06.2018 16:17, David Christensen wrote:
>>
>> I'm replying to my self. I just installed a new catalog under the same ic-instance, and there is still something strange. If i edit the locale-table via the admin-ui, all catalogs "freeze" for a certain period before the "lock/timeout" seems to get released and all pages work again smoothly.
>>
>> I'll try now with a fresh perlbrew-version of 5.24.1 and a fresh ic-install to see if i still have the same problem..
>
> When this is locked up is it responding to other requests? (You’d want to check in a different browser, as it locks sessions from the same session.) Basically trying to see if this is a local session-locking issue, or if it is a global IC lockup issue.

I tend to forget that this list puts the actoal sender in the from-field
instead of interchange-users@icdevgroup.org, so i replied (again :-)
directly to David.. So here again sent to the list :-)


Hi David,

It looks to be a global issue. I tested with two different browsers, and
each browser in private-/incognito-mode. When saving or opening a row
from the locale-table, all freezes for 1-2 min.

I'll install with perlbrew perl 5.10.1 (the same version as on
demo.icdevgroup.org), and let's see if the problem goes away.

René

_______________________________________________
interchange-users mailing list
interchange-users@icdevgroup.org
http://www.icdevgroup.org/mailman/listinfo/interchange-users
Re: IC freezes [ In reply to ]
On 26.06.2018 09:11, René wrote:
> Hi David,
>
> It looks to be a global issue. I tested with two different browsers, and
> each browser in private-/incognito-mode. When saving or opening a row
> from the locale-table, all freezes for 1-2 min.
>
> I'll install with perlbrew perl 5.10.1 (the same version as on
> demo.icdevgroup.org), and let's see if the problem goes away.

I installed a fresh perl via perlbrew (perl-5.10.1) and fresh copy of
IC, and the freezing seems now to be gone. I wonder what was causing
this issue..

René

_______________________________________________
interchange-users mailing list
interchange-users@icdevgroup.org
http://www.icdevgroup.org/mailman/listinfo/interchange-users
Re: IC freezes [ In reply to ]
As I said initially, I am sure this is a file locking issue. If you update
GDBM files, you are going to get locking problems while it is happening.
The locks stay on until the process ends, and this should happen for any
Interchange and Perl.

You will continue to have some problems as long as you are updating GDBM
databases from the running process. Move the DB to SQL and the problems
should disappear.

On Sun, Jul 8, 2018 at 12:13 PM, René <interchange@hertell.com> wrote:

> On 26.06.2018 09:11, René wrote:
>
>> Hi David,
>>
>> It looks to be a global issue. I tested with two different browsers, and
>> each browser in private-/incognito-mode. When saving or opening a row from
>> the locale-table, all freezes for 1-2 min.
>>
>> I'll install with perlbrew perl 5.10.1 (the same version as on
>> demo.icdevgroup.org), and let's see if the problem goes away.
>>
>
> I installed a fresh perl via perlbrew (perl-5.10.1) and fresh copy of IC,
> and the freezing seems now to be gone. I wonder what was causing this
> issue..
>
>
> René
>
> _______________________________________________
> interchange-users mailing list
> interchange-users@icdevgroup.org
> http://www.icdevgroup.org/mailman/listinfo/interchange-users
>



--
Just because something is obviously happening doesn't mean something
obvious is happening. --Larry Wall
Re: IC freezes [ In reply to ]
Also, there is an IC_LOCKING option. If you have that set for the table,
you'll also have trouble.

On Sun, Jul 8, 2018 at 5:12 PM, Mike Heins <mike@heins.com> wrote:

> As I said initially, I am sure this is a file locking issue. If you update
> GDBM files, you are going to get locking problems while it is happening.
> The locks stay on until the process ends, and this should happen for any
> Interchange and Perl.
>
> You will continue to have some problems as long as you are updating GDBM
> databases from the running process. Move the DB to SQL and the problems
> should disappear.
>
> On Sun, Jul 8, 2018 at 12:13 PM, René <interchange@hertell.com> wrote:
>
>> On 26.06.2018 09:11, René wrote:
>>
>>> Hi David,
>>>
>>> It looks to be a global issue. I tested with two different browsers, and
>>> each browser in private-/incognito-mode. When saving or opening a row from
>>> the locale-table, all freezes for 1-2 min.
>>>
>>> I'll install with perlbrew perl 5.10.1 (the same version as on
>>> demo.icdevgroup.org), and let's see if the problem goes away.
>>>
>>
>> I installed a fresh perl via perlbrew (perl-5.10.1) and fresh copy of IC,
>> and the freezing seems now to be gone. I wonder what was causing this
>> issue..
>>
>>
>> René
>>
>> _______________________________________________
>> interchange-users mailing list
>> interchange-users@icdevgroup.org
>> http://www.icdevgroup.org/mailman/listinfo/interchange-users
>>
>
>
>
> --
> Just because something is obviously happening doesn't mean something
> obvious is happening. --Larry Wall
>



--
Just because something is obviously happening doesn't mean something
obvious is happening. --Larry Wall
Re: IC freezes [ In reply to ]
On 09.07.2018 01:12, Mike Heins wrote:
> As I said initially, I am sure this is a file locking issue. If you
> update GDBM files, you are going to get locking problems while it is
> happening. The locks stay on until the process ends, and this should
> happen for any Interchange and Perl.
>
> You will continue to have some problems as long as you are updating GDBM
> databases from the running process. Move the DB to SQL and the problems
> should disappear.

Hi,

Isn't the idea with the gdbm-tables that they get loaded into memory
when IC is reloaded, and speeds up the translation-process?

If the same can be fixed with an sql-database, then this would be really
great!

René

_______________________________________________
interchange-users mailing list
interchange-users@icdevgroup.org
http://www.icdevgroup.org/mailman/listinfo/interchange-users
Re: IC freezes [ In reply to ]
No. GDBM databases are tied to GDBM files, period.

ALL locale tables are loaded into memory when IC starts, no matter what
underlying DB structure they use. This doesn't mean that, if you update
them, that the rules for databases are changed.

There is also a MEMORY db type, but that has no advantage for Locale,
because, again, all locale tables are loaded into memory even if they are
SQL or GDBM.

On Mon, Jul 9, 2018 at 6:29 AM, René <interchange@hertell.com> wrote:

>
> On 09.07.2018 01:12, Mike Heins wrote:
>
>> As I said initially, I am sure this is a file locking issue. If you
>> update GDBM files, you are going to get locking problems while it is
>> happening. The locks stay on until the process ends, and this should happen
>> for any Interchange and Perl.
>>
>> You will continue to have some problems as long as you are updating GDBM
>> databases from the running process. Move the DB to SQL and the problems
>> should disappear.
>>
>
> Hi,
>
> Isn't the idea with the gdbm-tables that they get loaded into memory when
> IC is reloaded, and speeds up the translation-process?
>
> If the same can be fixed with an sql-database, then this would be really
> great!
>
> René
>
> _______________________________________________
> interchange-users mailing list
> interchange-users@icdevgroup.org
> http://www.icdevgroup.org/mailman/listinfo/interchange-users
>



--
Just because something is obviously happening doesn't mean something
obvious is happening. --Larry Wall
Re: IC freezes [ In reply to ]
On 09.07.2018 15:17, Mike Heins wrote:
> No. GDBM databases are tied to GDBM files, period.
>
> ALL locale tables are loaded into memory when IC starts, no matter what
> underlying DB structure they use. This doesn't mean that, if you update
> them, that the rules for databases are changed.
>
> There is also a MEMORY db type, but that has no advantage for Locale,
> because, again, all locale tables are loaded into memory even if they
> are SQL or GDBM.

This is great! I'll switch to sql right away :-) Is there a particular
reason why some tables are still tied to GDBM and moved to sql? Is it
that after editing the .txt-files, they get re-imporetd to GDBM?

René

_______________________________________________
interchange-users mailing list
interchange-users@icdevgroup.org
http://www.icdevgroup.org/mailman/listinfo/interchange-users
Re: IC freezes [ In reply to ]
On 09.07.2018 20:59, René wrote:

> This is great! I'll switch to sql right away :-) Is there a particular
> reason why some tables are still tied to GDBM and moved to sql? Is it
> that after editing the .txt-files, they get re-imporetd to GDBM?

I tried with a locale.pgsql that looks like this

Database locale locale.txt __SQLDSN__
Database locale DEFAULT_TYPE text
Database locale KEY code
Database locale COLUMN_DEF "code=text NOT NULL PRIMARY KEY"

But the import seems to fail reading UTF8-characters (eg. &auml; &ouml;
etc), and the result is a mess.. Is there something that i have missed
in the configfile?

René

_______________________________________________
interchange-users mailing list
interchange-users@icdevgroup.org
http://www.icdevgroup.org/mailman/listinfo/interchange-users
Re: IC freezes [ In reply to ]
On Mon, 9 Jul 2018, René wrote:

> I tried with a locale.pgsql that looks like this
>
> Database locale locale.txt __SQLDSN__
> Database locale DEFAULT_TYPE text
> Database locale KEY code
> Database locale COLUMN_DEF "code=text NOT NULL PRIMARY KEY"
>
> But the import seems to fail reading UTF8-characters (eg. &auml; &ouml; etc),
> and the result is a mess.. Is there something that i have missed in the
> configfile?

Do you have PG_ENABLE_UTF8 (IIRC you're using Postgres) set as
DatabaseDefault, or for that specific table?

If you can't get the import working, you can always import the data to
your locale table some other way since you'll never want Interchange to do
that again once it's in Postgres.

Jon


--
Jon Jensen
End Point Corporation
https://www.endpoint.com/
Re: IC freezes [ In reply to ]
On 09.07.2018 23:06, Jon Jensen wrote:

> Do you have PG_ENABLE_UTF8 (IIRC you're using Postgres) set as
> DatabaseDefault, or for that specific table?
>
> If you can't get the import working, you can always import the data to
> your locale table some other way since you'll never want Interchange to
> do that again once it's in Postgres.

Yes, i have PG_ENABLE_UTF8 set, so this is probably something else that
causes me problems. Great idea to go for importing straight into
postgres.. I'll go for that! :-)

René

_______________________________________________
interchange-users mailing list
interchange-users@icdevgroup.org
http://www.icdevgroup.org/mailman/listinfo/interchange-users
Re: IC freezes [ In reply to ]
On 09.07.2018 23:31, René wrote:
> On 09.07.2018 23:06, Jon Jensen wrote:
>
>> Do you have PG_ENABLE_UTF8 (IIRC you're using Postgres) set as
>> DatabaseDefault, or for that specific table?
>>
>> If you can't get the import working, you can always import the data to
>> your locale table some other way since you'll never want Interchange
>> to do that again once it's in Postgres.
>
> Yes, i have PG_ENABLE_UTF8 set, so this is probably something else that
> causes me problems. Great idea to go for importing straight into
> postgres.. I'll go for that! :-)

Importing as tab-separated via pgadmin in text-mode worked great :-)

René

_______________________________________________
interchange-users mailing list
interchange-users@icdevgroup.org
http://www.icdevgroup.org/mailman/listinfo/interchange-users