Mailing List Archive

Back Compatibility
I have been thinking for a while that is time we revisit our back-
compatibility "policy" (http://wiki.apache.org/lucene-java/BackwardsCompatibility
) in terms of maybe becoming a little leaner and also in terms of
addressing some issues that come up from time to time in relation to
bug fixes that effect how Tokens are produced. As examples of the
latter, see: https://issues.apache.org/jira/browse/LUCENE-1084 and https://issues.apache.org/jira/browse/LUCENE-1100
. Examples of the former issue include things like removing
deprecations sooner and the ability to add new methods to interfaces
(both of these are not to be done ad-hoc)

In the case of bugs, the main issue is that people may be expecting
the "incorrect" behavior (admittedly, the maxFieldLength is not
incorrect), so the question becomes should we be in the business of
preserving incorrect values for a full version?

In the case of being "leaner", there are times when it would be useful
to be able to add new methods to interfaces w/o waiting for a full
major release (other projects do this) and also being able to pare
down the deprecated methods sooner.

I propose a couple of solutions to the leaner issue, but I am not sure
how to handle the incorrectness issue, although I suppose it could be
similar. With all of this, I really think the issue comes down to how
we communicate current and future changes to our users.

1. We add a new section to CHANGES for each release, at the top where
we can declare what deprecations will be removed in the _next_ release
(major or minor) and also any interface API changes
2. When deprecating, the @deprecate tag should declare what version it
will be removed in and that version must be one greater than the next
targeted release. That is, if the next release is 2.4, then anything
deprecated in 2.3 is game to be removed in 2.9.
3. Other ways of communicating changes????

My reasoning for this solution: Our minor release cycles are
currently in the 3-6 months range and our major release cycles are in
the 1-1.5 year range. I think giving someone 4-8 (or whatever) months
is more than enough time to prepare for API changes. I am not sure
how this would effect Index changes, but I do think we should KEEP our
current index reading policy where possible. This may mean that some
deprecated items cannot be removed until a major release and I think
that is fine.

Do people think that the bug issue also fits into this way of doing
things? Or do we need another way to think about those?

These are just suggestions and I am interested in hearing more about
what people think. I know, in some sense, it may make us less stable,
but I doubt it given the time frame of our releases. I also know a
perfectly valid response is "If it ain't broke, don't fix it" and to
great extent, I know it ain't broke. And believe me, I am fine with
that. I am just wondering if there is an opportunity to make Lucene
better.

Cheers,
Grant


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Back Compatibility [ In reply to ]
> Examples of the former issue include things like removing
> deprecations sooner and the ability to add new methods to interfaces
> (both of these are not to be done ad-hoc)

What would be the difference between ad-hoc and non-ad-hoc?

Bill

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Back Compatibility [ In reply to ]
On Jan 17, 2008, at 2:42 PM, Bill Janssen wrote:

>> Examples of the former issue include things like removing
>> deprecations sooner and the ability to add new methods to interfaces
>> (both of these are not to be done ad-hoc)
>
> What would be the difference between ad-hoc and non-ad-hoc?
>

Maybe bad choice of words, but I meant to say that no interface/
deprecation changes would be done without announcing it and there
being at least one release in the meantime. Thus, if we wanted to add
isFancySchmancy() onto Fieldable today, it would have to be announced,
patch provided and referenced, a release without it (i.e. 2.3) and
then it would be available in 2.4. By ad-hoc, I meant that we
wouldn't just announce it and then have it show up in 2.3 and not give
people time to digest it.

HTH,
Grant


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Back Compatibility [ In reply to ]
Grant Ingersoll wrote:
> 1. We add a new section to CHANGES for each release, at the top where we
> can declare what deprecations will be removed in the _next_ release
> (major or minor) and also any interface API changes
> 2. When deprecating, the @deprecate tag should declare what version it
> will be removed in and that version must be one greater than the next
> targeted release. That is, if the next release is 2.4, then anything
> deprecated in 2.3 is game to be removed in 2.9.

This would mean that one could never simply drop in the new jar and
expect things to still work, which is something that we currently try to
guarantee. That's a significant thing to give up, in terms of
usability. In my experience, folks hate incompatible changes, since
they're frequently no longer actively developing the portion of the code
that's broken, aren't seeking the new feature, etc. This is why lots of
folks stay back on old versions.

In terms of benefits, this would permit us to evolve APIs more rapidly.
So it pits external usability against API evolution speed, with no
clear winner. +0

Doug

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Back Compatibility [ In reply to ]
Grant Ingersoll wrote:
>
> My reasoning for this solution: Our minor release cycles are
> currently in the 3-6 months range and our major release cycles are in
> the 1-1.5 year range. I think giving someone 4-8 (or whatever) months
> is more than enough time to prepare for API changes. I am not sure
> how this would effect Index changes, but I do think we should KEEP our
> current index reading policy where possible. This may mean that some
> deprecated items cannot be removed until a major release and I think
> that is fine.

Personally, I like the stability of Lucene.

I don't see any problems with deprecations being done earlier, but
actual removal still at the major release.

Is there a roadmap of changes from 3.0 to 4.0 that would warrant such a
procedural change? What is Lucene missing that would have such a change?

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Back Compatibility [ In reply to ]
On Jan 17, 2008, at 4:14 PM, Doug Cutting wrote:

> Grant Ingersoll wrote:
>> 1. We add a new section to CHANGES for each release, at the top
>> where we can declare what deprecations will be removed in the
>> _next_ release (major or minor) and also any interface API changes
>> 2. When deprecating, the @deprecate tag should declare what version
>> it will be removed in and that version must be one greater than the
>> next targeted release. That is, if the next release is 2.4, then
>> anything deprecated in 2.3 is game to be removed in 2.9.
>
> This would mean that one could never simply drop in the new jar and
> expect things to still work, which is something that we currently
> try to guarantee. That's a significant thing to give up, in terms
> of usability. In my experience, folks hate incompatible changes,
> since they're frequently no longer actively developing the portion
> of the code that's broken, aren't seeking the new feature, etc.
> This is why lots of folks stay back on old versions.
>

Yep. I agree. I don't make the suggestion lightly and a perfectly
valid answer is let's not bother. By the same token, do people
really just drop in a new release in these days of continuous
integration and short release cycles? At a minimum, it has to go
through a fair amount of testing, right?

An alternative is to do major releases more often, but, to some
extent, it's all just semantics. The key is communicating what the
exact changes are, regardless of what you call the version number.
This does bring an extra burden in that we would need to be better
about communicating upcoming changes. That I am not exactly thrilled
about, either.


> In terms of benefits, this would permit us to evolve APIs more
> rapidly. So it pits external usability against API evolution speed,
> with no clear winner. +0

Yep, I am still on the fence, but wanted to revisit the discussion in
light of some recent bugs and comments about using Interfaces more.

Perhaps more important is how to handle fixing issues that change how
a document is indexed. Do we preserve the incorrectness for the sake
of back-compatibility? Or do we tell them this way is flat out wrong
and it won't be supported anymore?

-Grant

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
RE: Back Compatibility [ In reply to ]
Hi Grant,

On 01/17/2008 at 7:51 AM, Grant Ingersoll wrote:
> Our minor release cycles are currently in the 3-6 months range
> and our major release cycles are in the 1-1.5 year range.

Since 2.0.0, including 2.3.0 - assuming it will be released in the next week or so - the minor release intervals will have averaged about 6.5 months, over three releases.

Historically, the major release cycle intervals have roughly been:

1.0 - 6 months (March 2000 - October 2000)
2.0.0 - 6 years (October 2000 - May 2006)

Six years is an incredibly long time to maintain backward compatibility.

Assuming there will be a 2.4 release, and then 3.0 following it, it's pretty optimistic (IMHO) to think that it will be released before June 2008, so for 3.0, that would be:

3.0.0 - 2 years (May 2006 - May 2008)

Two years doesn't seem so long in comparison :).

> I think giving someone 4-8 (or whatever) months is more than
> enough time to prepare for API changes. I am not sure how
> this would effect Index changes, but I do think we should
> KEEP our current index reading policy where possible. This
> may mean that some deprecated items cannot be removed until
> a major release and I think that is fine.

Given the 6.5 month average minor release interval for the most recent major release, and the relatively low probability that this will shrink appreciably, you seem in essense to be advocating altogether abandoning backward API compatibility from one (minor) release to the next.

However, below you are advocating a minimum of one "test balloon" release between incompatible changes:

On 01/17/2008 at 3:41 PM, Grant Ingersoll wrote:
> [N]o interface/deprecation changes would be done without announcing it
> and there being at least one release in the meantime. Thus, if we
> wanted to add isFancySchmancy() onto Fieldable today, it would have to
> be announced, patch provided and referenced, a release without it (i.e.
> 2.3) and then it would be available in 2.4. By ad-hoc, I meant that we
> wouldn't just announce it and then have it show up in 2.3 and not give
> people time to digest it.

If I understand you correctly, a major release series could contain a whole series of non-aligned overlapping back-incompatible changes, since you are allowing individual features to alter backward incompatibility independently of other features. I think this is actually worse than just abandoning back-compatibility, since users would have to look up information on each individual feature to be able to figure out whether they can do a drop-in upgrade.

Steve

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Back Compatibility [ In reply to ]
If they are " no longer actively developing the portion of the code
that's broken, aren't seeking the new feature, etc", and they stay
back on old versions... isn't that exactly what we want? They can
stay on the old version, and new application development uses the
newer version.

It would be different if it was a core JRE interface or similar -
this is an optional jar.

Part of what always made Windows so fragile is that as it evolved
they tried to maintain backward compatibility - making working with
the old/new code and fixing bugs almost impossible. The bloat became
impossible to deal with.

I bet, if you did a poll of all Lucene users, you would find a
majority of them still only run 1.4.3, or maybe 1.9. Even with 2.0,
2.3, or 3.0, that is still going to be the case.

As always, JMO.

On Jan 17, 2008, at 3:14 PM, Doug Cutting wrote:

> Grant Ingersoll wrote:
>> 1. We add a new section to CHANGES for each release, at the top
>> where we can declare what deprecations will be removed in the
>> _next_ release (major or minor) and also any interface API changes
>> 2. When deprecating, the @deprecate tag should declare what
>> version it will be removed in and that version must be one greater
>> than the next targeted release. That is, if the next release is
>> 2.4, then anything deprecated in 2.3 is game to be removed in 2.9.
>
> This would mean that one could never simply drop in the new jar and
> expect things to still work, which is something that we currently
> try to guarantee. That's a significant thing to give up, in terms
> of usability. In my experience, folks hate incompatible changes,
> since they're frequently no longer actively developing the portion
> of the code that's broken, aren't seeking the new feature, etc.
> This is why lots of folks stay back on old versions.
>
> In terms of benefits, this would permit us to evolve APIs more
> rapidly. So it pits external usability against API evolution
> speed, with no clear winner. +0
>
> Doug
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Back Compatibility [ In reply to ]
On Jan 17, 2008, at 7:57 PM, robert engels wrote:

> If they are " no longer actively developing the portion of the code
> that's broken, aren't seeking the new feature, etc", and they stay
> back on old versions... isn't that exactly what we want? They can
> stay on the old version, and new application development uses the
> newer version.
>
> It would be different if it was a core JRE interface or similar -
> this is an optional jar.
>
> Part of what always made Windows so fragile is that as it evolved
> they tried to maintain backward compatibility - making working with
> the old/new code and fixing bugs almost impossible. The bloat became
> impossible to deal with.
>
> I bet, if you did a poll of all Lucene users, you would find a
> majority of them still only run 1.4.3, or maybe 1.9. Even with 2.0,
> 2.3, or 3.0, that is still going to be the case.

I found that upgrading from 1.4.3 (our first version) to 1.9, to
2.0, ... 2.2 and even 2.3 rc was painless.

The deprecations in 1.9 gave clear guidance on how to do the upgrade.
Very easy to do. And with Lucene's robust test suite, I had great
confidence that it would work without much testing.

Going forward was simply a matter of dropping in the new jar and
enjoying the improved performance.

The forward compatibility of the actual index was a great boon.

So, while one may not be actively developing code, dropping in a new
jar and getting huge performance gains is a great plus.

Many thanks to you all for such a stable product.

-- DM Smith


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Back Compatibility [ In reply to ]
I guess I am suggesting that instead of maintaining the whole major/
minor thing (not including file format) that we relax a bit and say
that any give feature we choose to remove or add has to go through two
release cycles, which according to your averages, would equal just
over 1 year's time. If people can't adapt to a coming change that was
announced a year ago, then I have to wonder why they need to upgrade
at all. (OK, that is a little strong, but...)

And mind you, I am not set in stone about this, I'm just starting the
conversation, being a bit of a devils advocate. I would like to know
if there is a way we can retain Lucene's solid stability and maturity
and encourage new and better features and not have to maintain so much
deprecated code.

I especially think this shorter cycle is useful when it comes to
deprecated features (not that there is much difference between
removing methods and adding methods.) Basically, we have code in
Lucene that we all agree should not be used, yet we leave it sitting
in there for, on average 2 years or more. Dropping that to be 1 year,
on average, is not going to all of a sudden break everyone and make
Lucene unstable. Besides, if people like the deprecated methods so
much, why upgrade in the first place? No one is forcing them.
Usually, the answer is there is some new feature somewhere else that
is needed, but that just shows that people are willing to invest the
time to get better features in the first place. Besides, just because
we can remove things, doesn't mean we have to remove them. For
instance, some bigger features that we improve, we may want to
deprecate for more than one full release cycle.

Fieldable is, in my mind a prime example of needing the ability to
announce additions. Let's say we come up with some new-fangled Field
type called Magic. This thing is so beautiful we all wonder how we
ever lived without it. Great. Now all we need to do is add an
isMagic() declaration onto Fieldable and we're good. Oops, can't do
that. Gotta wait 2 more years for 4.0. Seriously, we have to be
locked into an interface for 2 years or more? And oh by the way, in
that 2 years, Lucene has been left in the dust b/c every other open
source search engine out there already has Magic capabilities.
Furthermore, that gives us one 6 month window (+/-) to get it right
for the next 2 years. I know, it's a bit over the top, but I think it
demonstrates the point. I also don't see what is unstable about
telling the community, well in advance, that the following API changes
are coming, please plan accordingly. Most projects out there don't
even do that. Seriously, in Maven you get updates to functionality
without even knowing you are getting them and I am no where near
advocating for that.

And again, I still think the more pertinent issue that needs to be
addressed is how to better handle bugs in things like Tokenization,
etc. where people may have dependencies on broken functionality, but
haven't fully comprehended that they have such a dependency. I don't
think those fixes/deprecations should have to wait for a major release.

Does anyone have experience w/ how other open source projects deal
with this? Do they have explicit policies? Is it ad hoc? I know for
instance, in a certain library I was using that the difference between
what was announced as a beta and as 1.0 was quite different. Granted,
one could expect that a bit more out of something that was going from
0.X to 1.0, but still it was a pretty significant, more or less
unannounced change (unless you count following all the commit messages
as announcement) and it will require a decent amount of work to upgrade.


-Grant


On Jan 17, 2008, at 6:58 PM, Steven A Rowe wrote:

> Hi Grant,
>
> On 01/17/2008 at 7:51 AM, Grant Ingersoll wrote:
>> Our minor release cycles are currently in the 3-6 months range
>> and our major release cycles are in the 1-1.5 year range.
>
> Since 2.0.0, including 2.3.0 - assuming it will be released in the
> next week or so - the minor release intervals will have averaged
> about 6.5 months, over three releases.
>
> Historically, the major release cycle intervals have roughly been:
>
> 1.0 - 6 months (March 2000 - October 2000)
> 2.0.0 - 6 years (October 2000 - May 2006)
>
> Six years is an incredibly long time to maintain backward
> compatibility.
>
> Assuming there will be a 2.4 release, and then 3.0 following it,
> it's pretty optimistic (IMHO) to think that it will be released
> before June 2008, so for 3.0, that would be:
>
> 3.0.0 - 2 years (May 2006 - May 2008)
>
> Two years doesn't seem so long in comparison :).
>
>> I think giving someone 4-8 (or whatever) months is more than
>> enough time to prepare for API changes. I am not sure how
>> this would effect Index changes, but I do think we should
>> KEEP our current index reading policy where possible. This
>> may mean that some deprecated items cannot be removed until
>> a major release and I think that is fine.
>
> Given the 6.5 month average minor release interval for the most
> recent major release, and the relatively low probability that this
> will shrink appreciably, you seem in essense to be advocating
> altogether abandoning backward API compatibility from one (minor)
> release to the next.
>
> However, below you are advocating a minimum of one "test balloon"
> release between incompatible changes:
>
> On 01/17/2008 at 3:41 PM, Grant Ingersoll wrote:
>> [N]o interface/deprecation changes would be done without announcing
>> it
>> and there being at least one release in the meantime. Thus, if we
>> wanted to add isFancySchmancy() onto Fieldable today, it would have
>> to
>> be announced, patch provided and referenced, a release without it
>> (i.e.
>> 2.3) and then it would be available in 2.4. By ad-hoc, I meant
>> that we
>> wouldn't just announce it and then have it show up in 2.3 and not
>> give
>> people time to digest it.
>
> If I understand you correctly, a major release series could contain
> a whole series of non-aligned overlapping back-incompatible changes,
> since you are allowing individual features to alter backward
> incompatibility independently of other features. I think this is
> actually worse than just abandoning back-compatibility, since users
> would have to look up information on each individual feature to be
> able to figure out whether they can do a drop-in upgrade.
>
> Steve
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>

--------------------------
Grant Ingersoll
http://lucene.grantingersoll.com
http://www.lucenebootcamp.com

Lucene Helpful Hints:
http://wiki.apache.org/lucene-java/BasicsOfPerformance
http://wiki.apache.org/lucene-java/LuceneFAQ





---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Back Compatibility [ In reply to ]
On Jan 17, 2008, at 9:30 PM, DM Smith wrote:

>
> On Jan 17, 2008, at 7:57 PM, robert engels wrote:
>
>> If they are " no longer actively developing the portion of the code
>> that's broken, aren't seeking the new feature, etc", and they stay
>> back on old versions... isn't that exactly what we want? They can
>> stay on the old version, and new application development uses the
>> newer version.
>>
>> It would be different if it was a core JRE interface or similar -
>> this is an optional jar.
>>
>> Part of what always made Windows so fragile is that as it evolved
>> they tried to maintain backward compatibility - making working with
>> the old/new code and fixing bugs almost impossible. The bloat
>> became impossible to deal with.
>>
>> I bet, if you did a poll of all Lucene users, you would find a
>> majority of them still only run 1.4.3, or maybe 1.9. Even with 2.0,
>> 2.3, or 3.0, that is still going to be the case.
>
> I found that upgrading from 1.4.3 (our first version) to 1.9, to
> 2.0, ... 2.2 and even 2.3 rc was painless.
>
> The deprecations in 1.9 gave clear guidance on how to do the
> upgrade. Very easy to do. And with Lucene's robust test suite, I had
> great confidence that it would work without much testing.

And I don't think this would change at all with what I am proposing.
We still would be giving clear guidance, we would just be saying it's
going to happen in 1 year, not 2 to 6.

>
>
> Going forward was simply a matter of dropping in the new jar and
> enjoying the improved performance.
>
> The forward compatibility of the actual index was a great boon.
>
> So, while one may not be actively developing code, dropping in a new
> jar and getting huge performance gains is a great plus.
>

But even going from 2.2. to 2.3, you get even bigger gains by doing
some work and actually taking the time to update your Analysis
process, reuse tokens, etc.


-Grant

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Back Compatibility [ In reply to ]
18 jan 2008 kl. 03.39 skrev Grant Ingersoll:

> Does anyone have experience w/ how other open source projects deal
> with this?

Would be a pain to implement, but it could be done as libcompat.

lucene-2.4-compat-core-3.0.jar


--
karl

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Back Compatibility [ In reply to ]
That brings us back to an earlier discussion: "if majority want to
break compatibility, then we should do so, and the minority can back-
port the changes to a previous release if they feel it is warranted."

I don't understand why that isn't a viable approach.

I agree that maintaining interface compatibility through versions is
a great ideal, but when the API becomes so bloated (deprecated
methods, and even usage patterns), it is much harder to learn, and
use properly.

Look at similar problems and how they handled in the JDK. The Date
class has been notorious since its inception. The Calendar class is
almost no better, now they are developing JSR-310 to replace both.

Existing code can still use the Date or Calendar classes. Both they
don't get any "newer" features. This would be similar to use the old
lucene jar.


On Jan 18, 2008, at 12:31 AM, Karl Wettin wrote:

>
> 18 jan 2008 kl. 03.39 skrev Grant Ingersoll:
>
>> Does anyone have experience w/ how other open source projects deal
>> with this?
>
> Would be a pain to implement, but it could be done as libcompat.
>
> lucene-2.4-compat-core-3.0.jar
>
>
> --
> karl
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Back Compatibility [ In reply to ]
18 jan 2008 kl. 07.41 skrev robert engels:

> Look at similar problems and how they handled in the JDK. The Date
> class has been notorious since its inception. The Calendar class is
> almost no better, now they are developing JSR-310 to replace both.
>
> Existing code can still use the Date or Calendar classes. Both they
> don't get any "newer" features. This would be similar to use the old
> lucene jar.

Sort of keeping all version in the trunk at once? IndexWriter2 is
IndexWriter with some some features replaced with something better?
And then IndexWriter3..? That's a bit messy if you ask me. But it
would work. But terrible messy.

--
karl

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Back Compatibility [ In reply to ]
That wasn't what I was thinking. They would use lucene23.jar if they
wanted the 2.3 API. Newer code uses the lucene30.jar for the 3.0 API.

The others could continue to back-port 3.0 features to 2.3.X if they
wished (and could do so without changing the API - private changes
only).

I think you can look at the Oracle JDBC drivers as an example. They
warn that a API is going away in the next release, then it is gone.
Yet the new drivers may perform much better (not to mention fix a lot
of bugs). If you weren't using the old features you can easily move
to the new jar, if not, you need to change your code. Granted, it is
much better now, since they no longer have many needed proprietary
features, and rely mostly on the JDBC specification. They often
release newer versions of earlier releases when critical bugs have
been fixed.

JDBC is another good example. JDBC 3.0 requires Java 5. You cannot
use JDBC 3.0 features without it. Because of this, many of the db
vendors latest drivers are Java 5 only, and you need to use a
previous release if running under 1.4.

On Jan 18, 2008, at 1:04 AM, Karl Wettin wrote:

>
> 18 jan 2008 kl. 07.41 skrev robert engels:
>
>> Look at similar problems and how they handled in the JDK. The Date
>> class has been notorious since its inception. The Calendar class
>> is almost no better, now they are developing JSR-310 to replace both.
>>
>> Existing code can still use the Date or Calendar classes. Both
>> they don't get any "newer" features. This would be similar to use
>> the old lucene jar.
>
> Sort of keeping all version in the trunk at once? IndexWriter2 is
> IndexWriter with some some features replaced with something better?
> And then IndexWriter3..? That's a bit messy if you ask me. But it
> would work. But terrible messy.
>
> --
> karl
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
RE: Back Compatibility [ In reply to ]
> Sort of keeping all version in the trunk at once? IndexWriter2 is
> IndexWriter with some some features replaced with something better?
> And then IndexWriter3..? That's a bit messy if you ask me. But it
> would work. But terrible messy.

Brrr, I hate this. Microsoft does this always when they update their COM
interfaces...

Uwe


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Back Compatibility [ In reply to ]
Me too...

On Jan 18, 2008, at 4:33 AM, Uwe Schindler wrote:

>> Sort of keeping all version in the trunk at once? IndexWriter2 is
>> IndexWriter with some some features replaced with something better?
>> And then IndexWriter3..? That's a bit messy if you ask me. But it
>> would work. But terrible messy.
>
> Brrr, I hate this. Microsoft does this always when they update their
> COM
> interfaces...
>
> Uwe
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Back Compatibility [ In reply to ]
: If they are " no longer actively developing the portion of the code that's
: broken, aren't seeking the new feature, etc", and they stay back on old
: versions... isn't that exactly what we want? They can stay on the old version,
: and new application development uses the newer version.

This basically mirrors a philosophy that is rising in the Perl
community evangelized by (a really smart dude named chromatic) ...
"why are we worry about the effect of upgrades on users who don't upgrade?"

The problem is not all users are created equal and not all users upgrade
for the same reasons or at the same time...

Group A: If someone is paranoid about upgrading, and is still running
lucene1.4.3 because they are afraid if they upgrade their app will break
and they don't want to deal with it; they don't care about known bugs in
lucene1.4.3, as long as those bugs haven't impacted them yet -- these
people aren't going to care wether we add a bunch of new methods to
interfaces, or remove a bunch of public methods from arbitrary releases,
because they are never going to see them. They might do a total rewrite
of their project later, and they'll worry about it then (when they have
lots of time and QA resources)

Group: B: At the other extreme, are the "free-spirited" developers (god i
hate that that the word "agile" has been co-opted) who are always eager to
upgrade to get the latest bells and whistles, and don't mind making
changes to code and recompiling everytime they upgrades -- just as long as
there are some decent docs on what to change.

Croup: C: In the middle is a larg group of people who are interested in
upgrading, who want bug fixes, are willing to write new code to take
advantage of new features, in some cases are even willing to make
small or medium changes their code to get really good performance
improvements ... but they don't have a lot of time or energy to constantly
rewrite big chunks of their app. For these people, knowing that they can
"drop in" the new version and it will work is a big reason why there are
willing to upgrade, and why they are willing to spend soem time
tweaking code to take advantage of the new features and the new
performacne enhaced APIs -- becuase they don't have to spend a lot of time
just to get the app working as well as it was before.

To draw an analogy...

Group A will stand in one place for a really long time no matter how easy
the path is. Once in a great while they will decide to march forward
dozens of miles in one big push, but only once they feel they have
adequate resources to make the entire trip at once.

Group B likes to frolic, and will happily take two sptens backward and
then 3 steps forward every day.

Group C will walk forward with you at a steady pace, and occasionally even
take a step back before moving forward, but only if the path is clear and
not very steap.

: I bet, if you did a poll of all Lucene users, you would find a majority of
: them still only run 1.4.3, or maybe 1.9. Even with 2.0, 2.3, or 3.0, that is
: still going to be the case.

That's probably true, but a nice perk of our current backwards
compatibility commitments is that when people pop up asking questions
about 1.4.3, we can give them like "upgrading to 2.0.0 solves your
problem" and that advice isn't a death sentence -- the steps to move
forward are small and easy.

I look at things the way things like Maven v1 vs v2 worked out, and how
that fractured the community for a long time (as far as i can tell it's
still pretty fractured) because the path from v1 to v2 was so steep and
involved backtracking so much and i worry that if we make changes to our
"copatibility pledge" that don't allow for an even forward walk, we'll
wind up with a heavily fractured community.



-Hoss


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Back Compatibility [ In reply to ]
: I guess I am suggesting that instead of maintaining the whole major/minor
: thing (not including file format) that we relax a bit and say that any give
: feature we choose to remove or add has to go through two release cycles, which
: according to your averages, would equal just over 1 year's time. If people
: can't adapt to a coming change that was announced a year ago, then I have to
: wonder why they need to upgrade at all. (OK, that is a little strong, but...)

as someone else pointed out somewhere in this thread (reading it all at
once i'm lossing track) this will make it harder for people to understand
how much effort it will be to go from version 3.4 to 3.5 ... is that a
drop in replacement?

Perhaps the crux of the issue is that we as a community need to become
more willing to crank out "major" releases ... if we just released 3.0 and
now someone came up with the "Magic" field type and it's really magically
and we want to start using it but it's not backwards compatibly -- well i
guess are next release just needs to be called 4.0 then ... it's clear
from the version number that this is a significant change, evne if it does
wind up getting released 3 months after v3.0

: And again, I still think the more pertinent issue that needs to be addressed
: is how to better handle bugs in things like Tokenization, etc. where people
: may have dependencies on broken functionality, but haven't fully comprehended
: that they have such a dependency. I don't think those fixes/deprecations
: should have to wait for a major release.

I think situations like this are the one place where using system
properties to force broken/legacy behavior would really make sense ... we
fix the code so all "new" users get the correct/better behavior, and we
document in the CHANGES.txt that the behavior has changed. the code is
drop in compatile for anyone who isn't relying on broken behavior, and if
you are you can set a system proberty to foce the old behavior.
(caveat: to support the few cases people have mentioned where you can't
set system properties easily (applets i think?) a static method should be
provided as well, so if you need old broken behavior *AND* you can't use
system properties you just have to add one line of code to your app)


: Does anyone have experience w/ how other open source projects deal with this?

Poorly.

The best solution I've seen is to support multiple "stable" branches.
we've talked about doing that before, but there haven't been any features
anyone has steped up to backport to an older version since that
discussion. (probably because we've done such a good job of making it easy
for people to upgrade)

As i mentioned elsewhere in this thread: i worry about the community
fragementing if we raise the bar on upgrading in order to lower the bar on
development ... having multiple "stable" branches seems like it could also
fragment the community very easily... people using 3.2.X releases not
being able to interact/help with people using 2.4.Y on the user list
because certain things work drasitcly differnetly.

backporting bug fixes is one thing, but i'm leary of backporting new
features and performance improvements (not that i would object to anyone
doing so ... i'm just scared of where it might lead)


-Hoss


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Back Compatibility [ In reply to ]
I don't think group C is interested in bug fixes. I just don't see
how Lucene is at all useful if the users are encountering any bug -
so they either don't use that feature, or they have already developed
a work-around (or they have patched the code in a way that avoids the
bug, yet is specific to their environment).

For example, I think the NFS work (bugs, fixes, etc.) was quite
substantial. I think the actual number of people trying to use NFS is
probably very low - as the initial implementation had so many
problems (and IMO is not a very good solution for distributed indexes
anyway). So all the work in trying to make NFS work "correctly"
behind the scenes may have been inefficient, since a more direct, yet
major fix may have solved the problem better (like distributed server
support, not shared index access).

I just think that trying to maintain API compatibility through major
releases is a bad idea. Leads to bloat, and complex code - both
internal and external. In order to achieve great gains in usability
and/or performance in a mature product like Lucene almost certainly
requires massive changes to the processes, algorithms and structures,
and the API should change as well to reflect this.

On Jan 22, 2008, at 2:30 PM, Chris Hostetter wrote:

>
> : If they are " no longer actively developing the portion of the
> code that's
> : broken, aren't seeking the new feature, etc", and they stay back
> on old
> : versions... isn't that exactly what we want? They can stay on the
> old version,
> : and new application development uses the newer version.
>
> This basically mirrors a philosophy that is rising in the Perl
> community evangelized by (a really smart dude named chromatic) ...
> "why are we worry about the effect of upgrades on users who don't
> upgrade?"
>
> The problem is not all users are created equal and not all users
> upgrade
> for the same reasons or at the same time...
>
> Group A: If someone is paranoid about upgrading, and is still running
> lucene1.4.3 because they are afraid if they upgrade their app will
> break
> and they don't want to deal with it; they don't care about known
> bugs in
> lucene1.4.3, as long as those bugs haven't impacted them yet -- these
> people aren't going to care wether we add a bunch of new methods to
> interfaces, or remove a bunch of public methods from arbitrary
> releases,
> because they are never going to see them. They might do a total
> rewrite
> of their project later, and they'll worry about it then (when they
> have
> lots of time and QA resources)
>
> Group: B: At the other extreme, are the "free-spirited" developers
> (god i
> hate that that the word "agile" has been co-opted) who are always
> eager to
> upgrade to get the latest bells and whistles, and don't mind making
> changes to code and recompiling everytime they upgrades -- just as
> long as
> there are some decent docs on what to change.
>
> Croup: C: In the middle is a larg group of people who are
> interested in
> upgrading, who want bug fixes, are willing to write new code to take
> advantage of new features, in some cases are even willing to make
> small or medium changes their code to get really good performance
> improvements ... but they don't have a lot of time or energy to
> constantly
> rewrite big chunks of their app. For these people, knowing that
> they can
> "drop in" the new version and it will work is a big reason why
> there are
> willing to upgrade, and why they are willing to spend soem time
> tweaking code to take advantage of the new features and the new
> performacne enhaced APIs -- becuase they don't have to spend a lot
> of time
> just to get the app working as well as it was before.
>
> To draw an analogy...
>
> Group A will stand in one place for a really long time no matter
> how easy
> the path is. Once in a great while they will decide to march forward
> dozens of miles in one big push, but only once they feel they have
> adequate resources to make the entire trip at once.
>
> Group B likes to frolic, and will happily take two sptens backward and
> then 3 steps forward every day.
>
> Group C will walk forward with you at a steady pace, and
> occasionally even
> take a step back before moving forward, but only if the path is
> clear and
> not very steap.
>
> : I bet, if you did a poll of all Lucene users, you would find a
> majority of
> : them still only run 1.4.3, or maybe 1.9. Even with 2.0, 2.3, or
> 3.0, that is
> : still going to be the case.
>
> That's probably true, but a nice perk of our current backwards
> compatibility commitments is that when people pop up asking questions
> about 1.4.3, we can give them like "upgrading to 2.0.0 solves your
> problem" and that advice isn't a death sentence -- the steps to move
> forward are small and easy.
>
> I look at things the way things like Maven v1 vs v2 worked out, and
> how
> that fractured the community for a long time (as far as i can tell
> it's
> still pretty fractured) because the path from v1 to v2 was so steep
> and
> involved backtracking so much and i worry that if we make changes
> to our
> "copatibility pledge" that don't allow for an even forward walk, we'll
> wind up with a heavily fractured community.
>
>
>
> -Hoss
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Back Compatibility [ In reply to ]
One more example on this. A lot of work was done on transaction
support. I would argue that this falls way short of what is needed,
since there is no XA transaction support. Since the lucene index
(unless stored in an XA db) is a separate resource, it really needs
XA support in order to be consistent with the other resources.

All of the transaction work that has been performed only guarantees
that barring a physical hardware failure the lucene index can be
opened and used at a known state. This index though is probably not
consistent with the other resources.

All that was done is that we can now guarantee that the index is
consistent at SOME point in time.

Given the work that was done, we are probably closer to adding XA
support, but I think this would be much easier if the concept of a
transaction was made first class through the API (and then XA
transactions need to be supported).

On Jan 22, 2008, at 2:49 PM, robert engels wrote:

> I don't think group C is interested in bug fixes. I just don't see
> how Lucene is at all useful if the users are encountering any bug -
> so they either don't use that feature, or they have already
> developed a work-around (or they have patched the code in a way
> that avoids the bug, yet is specific to their environment).
>
> For example, I think the NFS work (bugs, fixes, etc.) was quite
> substantial. I think the actual number of people trying to use NFS
> is probably very low - as the initial implementation had so many
> problems (and IMO is not a very good solution for distributed
> indexes anyway). So all the work in trying to make NFS work
> "correctly" behind the scenes may have been inefficient, since a
> more direct, yet major fix may have solved the problem better (like
> distributed server support, not shared index access).
>
> I just think that trying to maintain API compatibility through
> major releases is a bad idea. Leads to bloat, and complex code -
> both internal and external. In order to achieve great gains in
> usability and/or performance in a mature product like Lucene almost
> certainly requires massive changes to the processes, algorithms and
> structures, and the API should change as well to reflect this.
>
> On Jan 22, 2008, at 2:30 PM, Chris Hostetter wrote:
>
>>
>> : If they are " no longer actively developing the portion of the
>> code that's
>> : broken, aren't seeking the new feature, etc", and they stay back
>> on old
>> : versions... isn't that exactly what we want? They can stay on
>> the old version,
>> : and new application development uses the newer version.
>>
>> This basically mirrors a philosophy that is rising in the Perl
>> community evangelized by (a really smart dude named chromatic) ...
>> "why are we worry about the effect of upgrades on users who don't
>> upgrade?"
>>
>> The problem is not all users are created equal and not all users
>> upgrade
>> for the same reasons or at the same time...
>>
>> Group A: If someone is paranoid about upgrading, and is still running
>> lucene1.4.3 because they are afraid if they upgrade their app will
>> break
>> and they don't want to deal with it; they don't care about known
>> bugs in
>> lucene1.4.3, as long as those bugs haven't impacted them yet -- these
>> people aren't going to care wether we add a bunch of new methods to
>> interfaces, or remove a bunch of public methods from arbitrary
>> releases,
>> because they are never going to see them. They might do a total
>> rewrite
>> of their project later, and they'll worry about it then (when they
>> have
>> lots of time and QA resources)
>>
>> Group: B: At the other extreme, are the "free-spirited" developers
>> (god i
>> hate that that the word "agile" has been co-opted) who are always
>> eager to
>> upgrade to get the latest bells and whistles, and don't mind making
>> changes to code and recompiling everytime they upgrades -- just as
>> long as
>> there are some decent docs on what to change.
>>
>> Croup: C: In the middle is a larg group of people who are
>> interested in
>> upgrading, who want bug fixes, are willing to write new code to take
>> advantage of new features, in some cases are even willing to make
>> small or medium changes their code to get really good performance
>> improvements ... but they don't have a lot of time or energy to
>> constantly
>> rewrite big chunks of their app. For these people, knowing that
>> they can
>> "drop in" the new version and it will work is a big reason why
>> there are
>> willing to upgrade, and why they are willing to spend soem time
>> tweaking code to take advantage of the new features and the new
>> performacne enhaced APIs -- becuase they don't have to spend a lot
>> of time
>> just to get the app working as well as it was before.
>>
>> To draw an analogy...
>>
>> Group A will stand in one place for a really long time no matter
>> how easy
>> the path is. Once in a great while they will decide to march forward
>> dozens of miles in one big push, but only once they feel they have
>> adequate resources to make the entire trip at once.
>>
>> Group B likes to frolic, and will happily take two sptens backward
>> and
>> then 3 steps forward every day.
>>
>> Group C will walk forward with you at a steady pace, and
>> occasionally even
>> take a step back before moving forward, but only if the path is
>> clear and
>> not very steap.
>>
>> : I bet, if you did a poll of all Lucene users, you would find a
>> majority of
>> : them still only run 1.4.3, or maybe 1.9. Even with 2.0, 2.3, or
>> 3.0, that is
>> : still going to be the case.
>>
>> That's probably true, but a nice perk of our current backwards
>> compatibility commitments is that when people pop up asking questions
>> about 1.4.3, we can give them like "upgrading to 2.0.0 solves your
>> problem" and that advice isn't a death sentence -- the steps to move
>> forward are small and easy.
>>
>> I look at things the way things like Maven v1 vs v2 worked out,
>> and how
>> that fractured the community for a long time (as far as i can tell
>> it's
>> still pretty fractured) because the path from v1 to v2 was so
>> steep and
>> involved backtracking so much and i worry that if we make changes
>> to our
>> "copatibility pledge" that don't allow for an even forward walk,
>> we'll
>> wind up with a heavily fractured community.
>>
>>
>>
>> -Hoss
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Back Compatibility [ In reply to ]
I humbly disagree about NFS. Arguing about where free time was invested,
or wasted, or inefficient, in an open source project just seems silly.
One of the great benefits is esoteric work that would normally not be
allowed for. NFS is easy. A lot of Lucene users don't care about Lucene.
They just want something easy to setup. It especially doesn't make send
when talking about Michael. He seems to spit out Lucene code in his
sleep. I doubt NFS stuff did anything but to make him more brilliant at
manipulating Lucene. It certainly hasn't made him any less prolific.

I am very in favor of your talk about transactional support. Man do I
want Lucene to have that. But the fact that we are getting to where the
index cannot be corrupted is still a great step forward. Knowing that my
indexes will not be corrupted while running at a place that needs access
24/7 is just wonderful. I can get something working for them quick,
whether its lost a bit of data or not. Now full support to guarantee
that my Lucene index is consistent with my Database? Even better. I
wish. But I am still very thankful for the first step of a guaranteed
consistent index.

Your glass is always half full ;) I aspire to your crankiness when I get
older.

- Mark


robert engels wrote:
> One more example on this. A lot of work was done on transaction
> support. I would argue that this falls way short of what is needed,
> since there is no XA transaction support. Since the lucene index
> (unless stored in an XA db) is a separate resource, it really needs XA
> support in order to be consistent with the other resources.
>
> All of the transaction work that has been performed only guarantees
> that barring a physical hardware failure the lucene index can be
> opened and used at a known state. This index though is probably not
> consistent with the other resources.
>
> All that was done is that we can now guarantee that the index is
> consistent at SOME point in time.
>
> Given the work that was done, we are probably closer to adding XA
> support, but I think this would be much easier if the concept of a
> transaction was made first class through the API (and then XA
> transactions need to be supported).
>
> On Jan 22, 2008, at 2:49 PM, robert engels wrote:
>
>> I don't think group C is interested in bug fixes. I just don't see
>> how Lucene is at all useful if the users are encountering any bug -
>> so they either don't use that feature, or they have already developed
>> a work-around (or they have patched the code in a way that avoids the
>> bug, yet is specific to their environment).
>>
>> For example, I think the NFS work (bugs, fixes, etc.) was quite
>> substantial. I think the actual number of people trying to use NFS is
>> probably very low - as the initial implementation had so many
>> problems (and IMO is not a very good solution for distributed indexes
>> anyway). So all the work in trying to make NFS work "correctly"
>> behind the scenes may have been inefficient, since a more direct, yet
>> major fix may have solved the problem better (like distributed server
>> support, not shared index access).
>>
>> I just think that trying to maintain API compatibility through major
>> releases is a bad idea. Leads to bloat, and complex code - both
>> internal and external. In order to achieve great gains in usability
>> and/or performance in a mature product like Lucene almost certainly
>> requires massive changes to the processes, algorithms and structures,
>> and the API should change as well to reflect this.
>>
>> On Jan 22, 2008, at 2:30 PM, Chris Hostetter wrote:
>>
>>>
>>> : If they are " no longer actively developing the portion of the
>>> code that's
>>> : broken, aren't seeking the new feature, etc", and they stay back
>>> on old
>>> : versions... isn't that exactly what we want? They can stay on the
>>> old version,
>>> : and new application development uses the newer version.
>>>
>>> This basically mirrors a philosophy that is rising in the Perl
>>> community evangelized by (a really smart dude named chromatic) ...
>>> "why are we worry about the effect of upgrades on users who don't
>>> upgrade?"
>>>
>>> The problem is not all users are created equal and not all users
>>> upgrade
>>> for the same reasons or at the same time...
>>>
>>> Group A: If someone is paranoid about upgrading, and is still running
>>> lucene1.4.3 because they are afraid if they upgrade their app will
>>> break
>>> and they don't want to deal with it; they don't care about known
>>> bugs in
>>> lucene1.4.3, as long as those bugs haven't impacted them yet -- these
>>> people aren't going to care wether we add a bunch of new methods to
>>> interfaces, or remove a bunch of public methods from arbitrary
>>> releases,
>>> because they are never going to see them. They might do a total
>>> rewrite
>>> of their project later, and they'll worry about it then (when they have
>>> lots of time and QA resources)
>>>
>>> Group: B: At the other extreme, are the "free-spirited" developers
>>> (god i
>>> hate that that the word "agile" has been co-opted) who are always
>>> eager to
>>> upgrade to get the latest bells and whistles, and don't mind making
>>> changes to code and recompiling everytime they upgrades -- just as
>>> long as
>>> there are some decent docs on what to change.
>>>
>>> Croup: C: In the middle is a larg group of people who are interested in
>>> upgrading, who want bug fixes, are willing to write new code to take
>>> advantage of new features, in some cases are even willing to make
>>> small or medium changes their code to get really good performance
>>> improvements ... but they don't have a lot of time or energy to
>>> constantly
>>> rewrite big chunks of their app. For these people, knowing that
>>> they can
>>> "drop in" the new version and it will work is a big reason why there
>>> are
>>> willing to upgrade, and why they are willing to spend soem time
>>> tweaking code to take advantage of the new features and the new
>>> performacne enhaced APIs -- becuase they don't have to spend a lot
>>> of time
>>> just to get the app working as well as it was before.
>>>
>>> To draw an analogy...
>>>
>>> Group A will stand in one place for a really long time no matter how
>>> easy
>>> the path is. Once in a great while they will decide to march forward
>>> dozens of miles in one big push, but only once they feel they have
>>> adequate resources to make the entire trip at once.
>>>
>>> Group B likes to frolic, and will happily take two sptens backward and
>>> then 3 steps forward every day.
>>>
>>> Group C will walk forward with you at a steady pace, and
>>> occasionally even
>>> take a step back before moving forward, but only if the path is
>>> clear and
>>> not very steap.
>>>
>>> : I bet, if you did a poll of all Lucene users, you would find a
>>> majority of
>>> : them still only run 1.4.3, or maybe 1.9. Even with 2.0, 2.3, or
>>> 3.0, that is
>>> : still going to be the case.
>>>
>>> That's probably true, but a nice perk of our current backwards
>>> compatibility commitments is that when people pop up asking questions
>>> about 1.4.3, we can give them like "upgrading to 2.0.0 solves your
>>> problem" and that advice isn't a death sentence -- the steps to move
>>> forward are small and easy.
>>>
>>> I look at things the way things like Maven v1 vs v2 worked out, and how
>>> that fractured the community for a long time (as far as i can tell it's
>>> still pretty fractured) because the path from v1 to v2 was so steep and
>>> involved backtracking so much and i worry that if we make changes to
>>> our
>>> "copatibility pledge" that don't allow for an even forward walk, we'll
>>> wind up with a heavily fractured community.
>>>
>>>
>>>
>>> -Hoss
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Back Compatibility [ In reply to ]
On Jan 22, 2008, at 3:45 PM, Chris Hostetter wrote:
>
> Perhaps the crux of the issue is that we as a community need to become
> more willing to crank out "major" releases ... if we just released
> 3.0 and
> now someone came up with the "Magic" field type and it's really
> magically
> and we want to start using it but it's not backwards compatibly --
> well i
> guess are next release just needs to be called 4.0 then ... it's clear
> from the version number that this is a significant change, evne if
> it does
> wind up getting released 3 months after v3.0

To paraphrase a dead English guy: A rose by any other name is still
the same, right?

Basically, all the version number tick saves them from is having to
read the CHANGES file, right?

To some extent, I am proposing that we clean out the cruft once a
year. Consider it spring cleaning. If we want to mark it as a major
version, I am fine with that. Basically, if history is any
indication, this would mean our releases will look like (given our
avg. 6 mos release cycle):
3.0
3.1
4.0
4.1
5.0
5.1

Thus, the version numbers become meaningless; the question is what do
we see as best for Lucene? We could just as easily call it Lucene
Summer '08 and Lucene Winter '08. Heck, we could pull the old MS Word
2.0 to MS Word 6.0 and jump to Lucene 6.0 next, too, for all I care.
I think 1 year is plenty long to keep both user Group B and C happy (A
will be oblivious). Once a year cleanup of code, in my mind is not
too burdensome for those in Group C. I consider myself in Group C,
for most tools I use (Lucene is probably the exception) and I can't
recall the last time I had an application that uses Lucene like stuff
that I haven't touched in over a year. But even for those other
tools, I expect that I am going to have a major upgrade once a year.
In fact, it is often part of the license agreement from commercial
companies and I would feel cheated if I didn't get it.

I think one could even argue that Group C would be happier w/ more
frequent removals of cruft, since it can be handled in a more
incremental way, versus an all at once upgrade every 2 years. They
have the option of how big of a chunk to bite off.


-Grant




---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Back Compatibility [ In reply to ]
Grant Ingersoll wrote:
> Does anyone have experience w/ how other open source projects deal with
> this?

Use abstract base classes instead of interfaces: they're much easier to
evolve back-compatibly. In Hadoop, for example, we really wish that
Mapper and Reducer were not interfaces and are very happy that
FileSystem is an abstract class.

Doug

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Back Compatibility [ In reply to ]
: To paraphrase a dead English guy: A rose by any other name is still the same,
: right?
:
: Basically, all the version number tick saves them from is having to read the
: CHANGES file, right?

Correct: i'm not disagreeing with your basic premise, just pointing out
that it can be done with the current model, and that predicable "version
identifiers" are a good idea when dealing with backwards compatibility.

: Thus, the version numbers become meaningless; the question is what do we see
: as best for Lucene? We could just as easily call it Lucene Summer '08 and
: Lucene Winter '08. Heck, we could pull the old MS Word 2.0 to MS Word 6.0 and

well .. i would argue that with what you hpothozied *then* version numbers
would becoming meaningless ... having 3.0, 3.1, 3.2, 4.0 would be no
differnet then having 3, 4, 5, 6 -- our version numbers would be
identifiers with no other context ... i'm just saying we should keep the
context in so that you know whether or not version X is backwards
compatible with version Y.

Which is not to say that we shouldn't hcange our version number format...

Ie: we could start using quad-tuple version numbers: 3.2.5.0 instead of 3.5.0

3: major version #
identifies file format back compatibility (as today)
2: api compat version #
classes/methods may be removed when this changes
5: minor version #
new methods may be added when this changes (as today)
0: patch version #
changes only when there are serious bug fixes

...that might mean that our version numbers go...

3.0.0.0
3.0.1.0
3.1.0.0
3.1.1.0
3.1.2.0
3.2.0.0

...where most numbers never get above "2" but at least the version number
conveys useful compatibility information (at no added developer "cost")



-Hoss


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org

1 2 3  View All