Mailing List Archive

Bill,
>
Great questions and thoughts, I've inlined comment below.  Sorry for the odd formatting, yahoo email just seems to get worse and worse...
>
>________________________________
>From: Bill Moseley <moseley@hank.org>
>To: The elegant MVC web framework <catalyst@lists.scsys.co.uk>
>Sent: Tuesday, September 17, 2013 10:52 AM
>Subject: [Catalyst] REST and versioning
>
>
>I've once again used up an hour or so reading Stack Overflow and blog posts on REST API versioning.   (And the arguments for and against.)
>
>
>Perhaps extending the discussion on how Catalyst supports REST:
>
>
>https://github.com/perl-catalyst/CatalystX-Proposals-REStandContentNegotiation
>
>
>I'm wondering if Catalyst might help in supporting API versions.  Somewhat similar to how Catalyst::Action::REST will call methods based on the HTTP method, perhaps call actions based on some version (provided by some means -- like a version in the path or in an Accept header).
>

People seem to get religious over how to version their API.  I doubt I'd want to take sides on this but here's how I think we could do both side (URL version and content type versioning

>
>Catalyst::Action::REST helps keep the actions tidy by calling methods specific to each method  (foo_GET, foo_PUT).  Obviously, we could simply check if ( $req->method eq 'GET' ) but would end up with pretty ugly actions and no automatic "Allow" header.
>
>
>With versions I'm concerned about that my foo_GET method will end up with a bunch of "if ( $version > 1.1 ) {....} elsif ($version >1.0 ) ...
>
>
>So, running with the C::Action::REST approach, something like:
>
>
>sub foo_GET { ... }  # Default
>sub foo_GET : Version( 1.1 ) { ... }  # Use if client requests version is 1.1
>
>
>Frankly, seems like maintenance nightmare and Action explosion.   Where that version comes from (url, Accpet header) is often debated (see links).
>
>

So the most recent stable Catalyst lets you declare http method matching natively, so here's how I might do this with Cat out of the box (untested code, but should serve the idea)

Lets say you want to match a url like /api/$version/...

package Myapp::Web::Controller::API;

use base 'Catalyst::Controller';

sub start : ChainedParent
 PathPrefix CaptureArgs(0)
{
  my ($self, $ctx) = @_;
}

  sub version_one : Chained('start') PathPart('1') Args(0) { ... }

  sub version_two : Chained('start') PathPart('2') Args(0) { ... }

1;

package Myapp::Web::Controller::API::1;

use base 'Catalyst::Controller';

sub start : ChainedParent
 PathPrefix CaptureArgs(0)
{
  my ($self, $ctx) = @_;
}

1;

package Myapp::Web::Controller::API::2;

use base 'Catalyst::Controller';

sub start : ChainedParent
 PathPrefix CaptureArgs(0)
{
  my ($self, $ctx) = @_;
}

1;

I guess you could use this with Catalyst:Action::REST based controllers as well.  There's probably a few ways you could do this. but I'd probably combine chaining with different ontrollers for different versions so that I could best group the common functionality.

If you wanted to take the content negotiation approach, this would fit right into the proposal

package MyApp::Web::Controller::User; use base 'Catalyst::Controller'; sub example :Local Provides('application/vnd.mycompany.user.v1+json') { my ($self, $ctx) = @_; } 1;
In this case the Provides attribute could be setup to match and route as expected.  We might want to consider allowing Regexp or some subset of regexp so that you could match more than one type of incoming requested response (for example you might care about the application/vnd.mycompany.user.v1 but not the JSON bit, and might use some other strategy, as best to avoid repeating yourself a lot.

>
>
>Any better ideas how to support versioning in Catalyst actions?
>
>

Well, I think the two general approaches are outlined here.  some people like to version as part of the URL, others follow a more purely restful approach and insist it is a matter for content negotiation.  I imagine you could have some plack middleware to smooth this over, for example to use content accept introspection in your code, but allow people to have some tag as a query param or similar.  We do this with the HTTP method matching for the newer Catalyst, since most browsers only support GET and POST method verbs, you can set a custom http header to map POST to PUT or DELETE.  This might be a good approach when dealing with clients that are not smart about doing true  RESTful negotiation.

It might also be useful to take a look at what some other frameworks in the RUby and Python world are doing.

>
>
>
>
>The subject of versioning is a bit overwhelming.  Here's some starting points, if curious:
>
>
>    * http://stackoverflow.com/questions/389169/best-practices-for-api-versioning
>
>
>    * http://www.lexicalscope.com/blog/2012/03/12/how-are-rest-apis-versioned/
>
>
>    * http://www.subbu.org/blog/2008/05/avoid-versioning-please
>
>
>    * http://stackoverflow.com/questions/2024600/rest-api-versioning-only-version-the-representation-not-the-resource-itself?lq=1
>
>
>    * http://www.informit.com/articles/article.aspx?p=1566460
>
>
>    * http://stackoverflow.com/questions/972226/how-to-version-rest-uris
>
>
>    * and plenty more...
>
>

Great links, and covers the based around a lot of these options.  Versioning needs is one reason I tend to not favor JSON for my canonical APIs, since its a great data format but lacks a bit for robustness as a document exchange format.  But that's another religious argument I guess :)

Thanks!

John

>
>
>--
>Bill Moseley
>moseley@hank.org
>_______________________________________________
>List: Catalyst@lists.scsys.co.uk
>Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
>Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
>Dev site: http://dev.catalyst.perl.org/
> 

_______________________________________________
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

On Tue, Sep 17, 2013 at 10:12 AM, John Napiorkowski <jjn1056@yahoo.com>wrote:

>
> People seem to get religious over how to version their API. I doubt I'd
> want to take sides on this but here's how I think we could do both side
> (URL version and content type versioning


Ya, I'm swayed by the Accept: header approach because in my mind
/api/v1/account/123 and /api/v2/account/123 seems like different resources.
(Well, I guess they are.) So, the versions are lazy way to make a new
resource location.

And even more so, seems like a dark path to go down. Is just that
individual resource versioned or is the entire API versioned? And if it's
the entire API, and the app needs to support multiple versions at the same
time, then need a way for methods to "fall-back" to v1 when only a few
methods change.

Or maybe the client would have to know which methods are v2 vs. v1 and
pick-and-choose.


Realistically, the problem that would likely come up is more related to
client versioning where an old client cannot support some new feature of
the API.

For example, say a service has a method to fetch a widget and a method to
list them.

GET /widget/123 # get widget 124
GET /widget # list all widgets.

So, some client app is designed to list the widgets and then fetch them
(for display or whatever).

Later, the service is upgraded and adds a new *type* of widget -- and that
is a type that the existing old client app cannot support.

Does the service need to know what the client can support?

sub widget_GET : Args(0) ... {
...
push @include_types, $new_widget_type if $client_version >= 1.1;


That's going to lead to some nasty spaghetti code over time.

Maybe not such a great example as one could argue here that the client
could GET /widget?type=1&type=2&type=3, but other changes might make that
not so easy.


In your chained example below how does that work with longer paths?

I'm trying to come up with a good example.....

/v1/document/1234 # fetch a document
/v1/document/1234/share # list who the document is shared with.

Then what happens if it's just the share method that has a new version?



> package Myapp::Web::Controller::API;
>
> use base 'Catalyst::Controller';
>
> sub start : ChainedParent
> PathPrefix CaptureArgs(0)
> {
> my ($self, $ctx) = @_;
> }
>
> sub version_one : Chained('start') PathPart('1') Args(0) { ... }
>
> sub version_two : Chained('start') PathPart('2') Args(0) { ... }
>
> 1;
>
> package Myapp::Web::Controller::API::1;
>
> use base 'Catalyst::Controller';
>
> sub start : ChainedParent
> PathPrefix CaptureArgs(0)
> {
> my ($self, $ctx) = @_;
> }
>
> 1;
>
> package Myapp::Web::Controller::API::2;
>
> use base 'Catalyst::Controller';
>
> sub start : ChainedParent
> PathPrefix CaptureArgs(0)
> {
> my ($self, $ctx) = @_;
> }
>
> 1;



Bill Moseley
moseley@hank.org

On Tue, Sep 17, 2013 at 10:12 AM, John Napiorkowski <jjn1056@yahoo.com> wrote:
>
>
>
>
>>People seem to get religious over how to version their API.  I doubt I'd want to take sides on this but here's how I think we could do both side (URL version and content type versioning 
>
>
>Ya, I'm swayed by the Accept: header approach because in my mind /api/v1/account/123 and /api/v2/account/123 seems like different resources.   (Well, I guess they are.)  So, the versions are lazy way to make a new resource location.
>

I think the REST people would say yes, but I know a lot of APIs use URL versioning since its easy (or seems so off the top) and I also know there are some people that think it is clearly and absolutely the correct way to do it (I worked with one in a previous job).  Luckily Catalyst chaining and method matching I think makes this easy to do off the top.

I still think having it 'restfully correct' in your core code and then using some middleware for compatibility with dumb clients is probably safe.  I generally think its not a bad idea to have middleware for your APIs that map file extensions to accept types and map a query param for versioning.  For example

http://mycompany.com/api/user.json?v=1

would map internally a request like

http:://mycompany.com/api/user
http-accept application/vnd.mycompany.user.v1+json

or thereabouts.

>
>And even more so, seems like a dark path to go down.  Is just that individual resource versioned or is the entire API versioned?   And if it's the entire API, and the app needs to support multiple versions at the same time, then need a way for methods to "fall-back" to v1 when only a few methods change.
>
>
>Or maybe the client would have to know which methods are v2 vs. v1 and pick-and-choose.
>
>

yeah its a dark path I also agree.  This is why content negotiation and hyperlink driven application state is the most forward looking, but its not easy to do right and honestly what most people think of when they think of a web API is just an endpoint that takes and servers json data.  Application state is not important to that view (and is the common view for APIs that are driving a bit of interactivity on a webpage.


>
>
>Realistically, the problem that would likely come up is more related to client versioning where an old client cannot support some new feature of the API.   
>

The idea with rest is that this should be part of the content negotiation.  A client asks for a particular acceptable media type and the server decides how best to respond.  At least for the common server side negotiation.  

>
>For example,  say a service has a method to fetch a widget and a method to list them.
>
>
>GET /widget/123  # get widget 124
>GET /widget  # list all widgets.
>
>
>So, some client app is designed to list the widgets and then fetch them (for display or whatever).
>
>
>Later, the service is upgraded and adds a new type of widget -- and that is a type that the existing old client app cannot support.
>


Ideally you could properly support this backwardly if possible.  Generally you have two types of changes, one where you add in stuff, and that should be such that old clients shouldn't die.  but sometimes you need to really change things and that can get you into trouble.  I've noted this is worse when people are lazy and just think serializing the data structure to json and sending it over the wire is enough.  Like you have a version one user that gets serialized to json like

{ 
  'name': 'john',
  'phone': '2222222222',
}

but then you suddenly realize you need more than one phone per person and so you do:

{ 
  'name': 'john',
  'phone': [1111111,2222222,333333]
}

and of course that breaks the client.  either you are stuck doing

{ 
  'name': 'john',
  'phone': '2222222222',
  'additional_phones': [ ... ]
}

if you want to preserve backcompat or you change the version.  My thinking would be to do both really.

Of course this is a place where it would have been smart for the developer to have used an existing standard that already thought of all this stuff.  But what I've seen is APIs tend to be lazily modeled.  People oftern just serialized the output of a database query because the tools make that easy and it seems fast.  But down the road you get a lot of pain.

funny enough, this is a case where the 'bloated' xml might have been better

<contact>
  <name>john</name>
  <phone>1111111</phone>
  <phone>22222222</phone>
</contact>

If the client is smart and parsing the xml with something like css style selectors (or even xpath) then both versions should work.  If you are lazy and using something like XML-Simple then of course it will go bang just like with json.  But again I'd be likely to want to use something like chard instead.

>
>Does the service need to know what the client can support?
>

The client ideally would specify in the http accept header what they want and the server can decide if its willing to play, or pick up the marbles and go home :)

>
>sub widget_GET : Args(0) ... {
>    ...
>    push @include_types, $new_widget_type if $client_version >= 1.1;
>
>
>
>
>That's going to lead to some nasty spaghetti code over time.
>
>
>Maybe not such a great example as one could argue here that the client could GET /widget?type=1&type=2&type=3, but other changes might make that not so easy.
>
>

I should go look for some canonically 'correct' REST application and port it.

>
>
>In your chained example below how does that work with longer paths?
>
>
>I'm trying to come up with a good example.....
>
>
>/v1/document/1234   # fetch a document
>/v1/document/1234/share  # list who the document is shared with.
>
>
>Then what happens if it's just the share method that has a new version?
>


Well, if its a method, then that's RPC not REST and all constraints are off the table!

I think Catalyst chaining does long paths pretty well, although something chaining gets a bit confusing.  

>
>
>
>package Myapp::Web::Controller::API;
>>
>>use base 'Catalyst::Controller';
>>
>>sub start : ChainedParent
>> PathPrefix CaptureArgs(0)
>>{
>>  my ($self, $ctx) = @_;
>>}
>>
>>  sub version_one : Chained('start') PathPart('1') Args(0) { ... }
>>
>>  sub version_two : Chained('start') PathPart('2') Args(0) { ... }
>>
>>1;
>>
>>package Myapp::Web::Controller::API::1;
>>
>>use base 'Catalyst::Controller';
>>
>>sub start : ChainedParent
>> PathPrefix CaptureArgs(0)
>>{
>>  my ($self, $ctx) = @_;
>>}
>>
>>1;
>>
>>package Myapp::Web::Controller::API::2;
>>
>>use base 'Catalyst::Controller';
>>
>>sub start : ChainedParent
>> PathPrefix CaptureArgs(0)
>>{
>>  my ($self, $ctx) = @_;
>>}
>>
>>1;
>
>
> Bill Moseley
>moseley@hank.org  

_______________________________________________
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/