Mailing List Archive

Hosting recommendations ... ?
Hey gang ...

Looking for a reputable (i.e., no hosting of spammers or other
ne'er-do-wells) hosting provider with possibly a global footprint. If
not, US is #1 desire; EU #2.

Requirements, more or less:

* Desire to host 2-3 hypervisors, probably running something akin to
Proxmox ...

* ~5-10TB storage with the possibility of expansion ...

* 1G hand-off / 100 Mbps or less commit ... i.e., low BW, but burstable.

* Bringing my own IP space and need to be able to peer BGP with vendor.

* Cross-DC redundancy or mirroring or somesuch desirable.

* Backups are of interest, although I can do my own if need be.

Any recommendations that are non-Amazonian? Feel free to reach out
off-list if you prefer.

#1 requirement ... the reputability part ...

Thank you all in advance!

- bryan
Re: Hosting recommendations ... ? [ In reply to ]
On Tue, Jan 19, 2021 at 7:45 AM Bryan Holloway <bryan@shout.net> wrote:
> Looking for a reputable (i.e., no hosting of spammers or other
> ne'er-do-wells) hosting provider with possibly a global footprint. If
> not, US is #1 desire; EU #2.
>

> * Desire to host 2-3 hypervisors, probably running something akin to
> Proxmox ...
> * ~5-10TB storage with the possibility of expansion ...

Howdy,

You should clarify what sort of hosting service you're looking for. A
normal cloud service won't see you running your own hypervisors. A
server farm will see you deploy your own hardware with whatever
storage you choose to install. Only a "bare metal" cloud service would
meet both of your listed requirements, where you lease both the
equipment and hosting service from the provider. However, combined
with your BGP requirement your options are very limited and expanding
storage usually means "lease a different one of our servers."

Regards,
Bill Herrin



--
Hire me! https://bill.herrin.us/resume/
Re: Hosting recommendations ... ? [ In reply to ]
I'm kind of confused when your concern is the reputability and yet you're
providing your own IP space.

It sounds more to me like you want to put 2-3 boxes in a data center. For
that pretty much any decent sized data center in any state would work for
the US.

Josh Luthman
24/7 Help Desk: 937-552-2340
Direct: 937-552-2343
1100 Wayne St
Suite 1337
Troy, OH 45373


On Tue, Jan 19, 2021 at 11:14 AM William Herrin <bill@herrin.us> wrote:

> On Tue, Jan 19, 2021 at 7:45 AM Bryan Holloway <bryan@shout.net> wrote:
> > Looking for a reputable (i.e., no hosting of spammers or other
> > ne'er-do-wells) hosting provider with possibly a global footprint. If
> > not, US is #1 desire; EU #2.
> >
>
> > * Desire to host 2-3 hypervisors, probably running something akin to
> > Proxmox ...
> > * ~5-10TB storage with the possibility of expansion ...
>
> Howdy,
>
> You should clarify what sort of hosting service you're looking for. A
> normal cloud service won't see you running your own hypervisors. A
> server farm will see you deploy your own hardware with whatever
> storage you choose to install. Only a "bare metal" cloud service would
> meet both of your listed requirements, where you lease both the
> equipment and hosting service from the provider. However, combined
> with your BGP requirement your options are very limited and expanding
> storage usually means "lease a different one of our servers."
>
> Regards,
> Bill Herrin
>
>
>
> --
> Hire me! https://bill.herrin.us/resume/
>
Re: Hosting recommendations ... ? [ In reply to ]
On Tue, Jan 19, 2021 at 8:20 AM Josh Luthman
<josh@imaginenetworksllc.com> wrote:
> I'm kind of confused when your concern is the reputability and yet you're providing your own IP space.

Hi Josh,

I'm not above discarding announcements with a "bulletproof" hoster in
the AS path. Are you?

Regards,
Bill Herrin

On Tue, Jan 19, 2021 at 8:20 AM Josh Luthman
<josh@imaginenetworksllc.com> wrote:
>
> I'm kind of confused when your concern is the reputability and yet you're providing your own IP space.
>
> It sounds more to me like you want to put 2-3 boxes in a data center. For that pretty much any decent sized data center in any state would work for the US.
>
> Josh Luthman
> 24/7 Help Desk: 937-552-2340
> Direct: 937-552-2343
> 1100 Wayne St
> Suite 1337
> Troy, OH 45373
>
>
> On Tue, Jan 19, 2021 at 11:14 AM William Herrin <bill@herrin.us> wrote:
>>
>> On Tue, Jan 19, 2021 at 7:45 AM Bryan Holloway <bryan@shout.net> wrote:
>> > Looking for a reputable (i.e., no hosting of spammers or other
>> > ne'er-do-wells) hosting provider with possibly a global footprint. If
>> > not, US is #1 desire; EU #2.
>> >
>>
>> > * Desire to host 2-3 hypervisors, probably running something akin to
>> > Proxmox ...
>> > * ~5-10TB storage with the possibility of expansion ...
>>
>> Howdy,
>>
>> You should clarify what sort of hosting service you're looking for. A
>> normal cloud service won't see you running your own hypervisors. A
>> server farm will see you deploy your own hardware with whatever
>> storage you choose to install. Only a "bare metal" cloud service would
>> meet both of your listed requirements, where you lease both the
>> equipment and hosting service from the provider. However, combined
>> with your BGP requirement your options are very limited and expanding
>> storage usually means "lease a different one of our servers."
>>
>> Regards,
>> Bill Herrin
>>
>>
>>
>> --
>> Hire me! https://bill.herrin.us/resume/



--
Hire me! https://bill.herrin.us/resume/
Re: Hosting recommendations ... ? [ In reply to ]
Fair questions -- answers in-line ...


On 1/19/21 5:19 PM, Josh Luthman wrote:
> I'm kind of confused when your concern is the reputability and yet
> you're providing your own IP space.

I care about the hosting environment's upstreams' opinions of the
downstream.

That is, I don't want to be in a situation where some jamoke (or
jamokes) does something which affects me from a collateral damage
standpoint.

> It sounds more to me like you want to put 2-3 boxes in a data center.
> For that pretty much any decent sized data center in any state would
> work for the US.

I would like to stop personally dealing with bare-metal. That's what I'm
doing now.
Re: Hosting recommendations ... ? [ In reply to ]
On Tue, Jan 19, 2021 at 8:31 AM Bryan Holloway <bryan@shout.net> wrote:
> I would like to stop personally dealing with bare-metal. That's what I'm
> doing now.

Hi Bryan,

Cloud = you get virtual servers with virtual storage, generally
adjustable to meet your needs. You manage the operating systems and
storage within the virtual environment. You DO NOT manage the host
operating systems or hypervisors.

Bare metal = you lease physical equipment. You manage all software on
the equipment including any hypervisors needed to run virtual servers.
You DO NOT deal with hardware break/fix, that problem belongs to the
service provider.

Colocation = You lease space in a data center. You provide physical
equipment in your custom configuration.

With this terminology, at least one of your requirements is unmeetable
for contradicting the others. So I ask again for clarification: which
of these do you seek?

Regards,
Bill Herrin
Re: Hosting recommendations ... ? [ In reply to ]
Perhaps I'm missing something, but in your #1 example "Cloud", what
prevents me from running a Proxmox ISO (which is more or less Debian)
vs. a "standard" Debian install on the provider's virtual server?

If I can, I've succeeded. That is the sort of hosting provider I'm
looking for, if they exist.

#2 would be suitable, but it seems to be that if leased bare-metal dies,
it will be some time for ETR. Less desirable, but I'm open to ideas.

#3 I do now. Trying to move away from that.


On 1/19/21 5:44 PM, William Herrin wrote:
> On Tue, Jan 19, 2021 at 8:31 AM Bryan Holloway <bryan@shout.net> wrote:
>> I would like to stop personally dealing with bare-metal. That's what I'm
>> doing now.
>
> Hi Bryan,
>
> Cloud = you get virtual servers with virtual storage, generally
> adjustable to meet your needs. You manage the operating systems and
> storage within the virtual environment. You DO NOT manage the host
> operating systems or hypervisors.
>
> Bare metal = you lease physical equipment. You manage all software on
> the equipment including any hypervisors needed to run virtual servers.
> You DO NOT deal with hardware break/fix, that problem belongs to the
> service provider.
>
> Colocation = You lease space in a data center. You provide physical
> equipment in your custom configuration.
>
> With this terminology, at least one of your requirements is unmeetable
> for contradicting the others. So I ask again for clarification: which
> of these do you seek?
>
> Regards,
> Bill Herrin
>
Re: Hosting recommendations ... ? [ In reply to ]
I'm not sure spammers and AS paths really go together.

Josh Luthman
24/7 Help Desk: 937-552-2340
Direct: 937-552-2343
1100 Wayne St
Suite 1337
Troy, OH 45373


On Tue, Jan 19, 2021 at 11:27 AM William Herrin <bill@herrin.us> wrote:

> On Tue, Jan 19, 2021 at 8:20 AM Josh Luthman
> <josh@imaginenetworksllc.com> wrote:
> > I'm kind of confused when your concern is the reputability and yet
> you're providing your own IP space.
>
> Hi Josh,
>
> I'm not above discarding announcements with a "bulletproof" hoster in
> the AS path. Are you?
>
> Regards,
> Bill Herrin
>
> On Tue, Jan 19, 2021 at 8:20 AM Josh Luthman
> <josh@imaginenetworksllc.com> wrote:
> >
> > I'm kind of confused when your concern is the reputability and yet
> you're providing your own IP space.
> >
> > It sounds more to me like you want to put 2-3 boxes in a data center.
> For that pretty much any decent sized data center in any state would work
> for the US.
> >
> > Josh Luthman
> > 24/7 Help Desk: 937-552-2340
> > Direct: 937-552-2343
> > 1100 Wayne St
> > Suite 1337
> > Troy, OH 45373
> >
> >
> > On Tue, Jan 19, 2021 at 11:14 AM William Herrin <bill@herrin.us> wrote:
> >>
> >> On Tue, Jan 19, 2021 at 7:45 AM Bryan Holloway <bryan@shout.net> wrote:
> >> > Looking for a reputable (i.e., no hosting of spammers or other
> >> > ne'er-do-wells) hosting provider with possibly a global footprint. If
> >> > not, US is #1 desire; EU #2.
> >> >
> >>
> >> > * Desire to host 2-3 hypervisors, probably running something akin to
> >> > Proxmox ...
> >> > * ~5-10TB storage with the possibility of expansion ...
> >>
> >> Howdy,
> >>
> >> You should clarify what sort of hosting service you're looking for. A
> >> normal cloud service won't see you running your own hypervisors. A
> >> server farm will see you deploy your own hardware with whatever
> >> storage you choose to install. Only a "bare metal" cloud service would
> >> meet both of your listed requirements, where you lease both the
> >> equipment and hosting service from the provider. However, combined
> >> with your BGP requirement your options are very limited and expanding
> >> storage usually means "lease a different one of our servers."
> >>
> >> Regards,
> >> Bill Herrin
> >>
> >>
> >>
> >> --
> >> Hire me! https://bill.herrin.us/resume/
>
>
>
> --
> Hire me! https://bill.herrin.us/resume/
>
Re: Hosting recommendations ... ? [ In reply to ]
I would assume that anyone providing a dedicated server has the means to facilitate timely hardware replacements. Ask for their SLA on that.




-----
Mike Hammett
Intelligent Computing Solutions
http://www.ics-il.com

Midwest-IX
http://www.midwest-ix.com

----- Original Message -----

From: "Bryan Holloway" <bryan@shout.net>
To: "William Herrin" <bill@herrin.us>
Cc: "NANOG list" <nanog@nanog.org>
Sent: Tuesday, January 19, 2021 11:18:03 AM
Subject: Re: Hosting recommendations ... ?

Perhaps I'm missing something, but in your #1 example "Cloud", what
prevents me from running a Proxmox ISO (which is more or less Debian)
vs. a "standard" Debian install on the provider's virtual server?

If I can, I've succeeded. That is the sort of hosting provider I'm
looking for, if they exist.

#2 would be suitable, but it seems to be that if leased bare-metal dies,
it will be some time for ETR. Less desirable, but I'm open to ideas.

#3 I do now. Trying to move away from that.


On 1/19/21 5:44 PM, William Herrin wrote:
> On Tue, Jan 19, 2021 at 8:31 AM Bryan Holloway <bryan@shout.net> wrote:
>> I would like to stop personally dealing with bare-metal. That's what I'm
>> doing now.
>
> Hi Bryan,
>
> Cloud = you get virtual servers with virtual storage, generally
> adjustable to meet your needs. You manage the operating systems and
> storage within the virtual environment. You DO NOT manage the host
> operating systems or hypervisors.
>
> Bare metal = you lease physical equipment. You manage all software on
> the equipment including any hypervisors needed to run virtual servers.
> You DO NOT deal with hardware break/fix, that problem belongs to the
> service provider.
>
> Colocation = You lease space in a data center. You provide physical
> equipment in your custom configuration.
>
> With this terminology, at least one of your requirements is unmeetable
> for contradicting the others. So I ask again for clarification: which
> of these do you seek?
>
> Regards,
> Bill Herrin
>
Re: Hosting recommendations ... ? [ In reply to ]
For #1, are you trying to do "Cloud-ception" e.g. running your own proxmox virtualization on top of an already virtual machine, so that you're basically two layers deep?

For #2, of course you need to be able to survive a hardware failure (using RAID1 or some flavour of DRBD for example) but having to think about such things is the "trade-off" of having access to the bare-metal layer.. it does have advantages, for example if you want to install your own virtualization layer without any involvement from the hosting provider. You'd usually have agreements with the hosting provider about how/when hardware replacements would be done.

Best regards,
Martijn
________________________________
From: NANOG <nanog-bounces+martijnschmidt=i3d.net@nanog.org> on behalf of Bryan Holloway <bryan@shout.net>
Sent: 19 January 2021 18:18
To: William Herrin <bill@herrin.us>
Cc: NANOG list <nanog@nanog.org>
Subject: Re: Hosting recommendations ... ?

Perhaps I'm missing something, but in your #1 example "Cloud", what
prevents me from running a Proxmox ISO (which is more or less Debian)
vs. a "standard" Debian install on the provider's virtual server?

If I can, I've succeeded. That is the sort of hosting provider I'm
looking for, if they exist.

#2 would be suitable, but it seems to be that if leased bare-metal dies,
it will be some time for ETR. Less desirable, but I'm open to ideas.

#3 I do now. Trying to move away from that.


On 1/19/21 5:44 PM, William Herrin wrote:
> On Tue, Jan 19, 2021 at 8:31 AM Bryan Holloway <bryan@shout.net> wrote:
>> I would like to stop personally dealing with bare-metal. That's what I'm
>> doing now.
>
> Hi Bryan,
>
> Cloud = you get virtual servers with virtual storage, generally
> adjustable to meet your needs. You manage the operating systems and
> storage within the virtual environment. You DO NOT manage the host
> operating systems or hypervisors.
>
> Bare metal = you lease physical equipment. You manage all software on
> the equipment including any hypervisors needed to run virtual servers.
> You DO NOT deal with hardware break/fix, that problem belongs to the
> service provider.
>
> Colocation = You lease space in a data center. You provide physical
> equipment in your custom configuration.
>
> With this terminology, at least one of your requirements is unmeetable
> for contradicting the others. So I ask again for clarification: which
> of these do you seek?
>
> Regards,
> Bill Herrin
>
Re: Hosting recommendations ... ? [ In reply to ]
On 1/19/21 11:44 AM, William Herrin wrote:
> Cloud = you get virtual servers with virtual storage, generally
> adjustable to meet your needs. You manage the operating systems and
> storage within the virtual environment. You DO NOT manage the host
> operating systems or hypervisors.

It's worth pointing out that nested virtualization is a thing these
days, and some providers might even support it! That means you could
buy one large instance and sub-divide it yourself into multiple VMs if
you want to.

In practice, unless you need that flexibility to dynamically spin the
VMs up and down with various specs AND don't want to or cannot use a
provider's API for that, I'm not sure why you'd want to if you didn't
have to for some crazy reason.
--
Brandon Martin
Re: Hosting recommendations ... ? [ In reply to ]
You make an excellent point, Martijn ... (and I suspect this is what
Bill was pointing out ...)

Virtualization on top of virtualization is inherently not the best idea.

I guess I'm looking for flexibility in the sense of being able to spin
up additional VMs at my leisure. In which case #2 could be suitable in
the right environment.

... in which case SLAs would be tantamount to success ...

So ... that said, any recommendations? :)


On 1/19/21 6:32 PM, Martijn Schmidt wrote:
> For #1, are you trying to do "Cloud-ception" e.g. running your own
> proxmox virtualization on top of an already virtual machine, so that
> you're basically two layers deep?
>
> For #2, of course you need to be able to survive a hardware failure
> (using RAID1 or some flavour of DRBD for example) but having to think
> about such things is the "trade-off" of having access to the bare-metal
> layer.. it does have advantages, for example if you want to install your
> own virtualization layer without any involvement from the hosting
> provider. You'd usually have agreements with the hosting provider about
> how/when hardware replacements would be done.
>
> Best regards,
> Martijn
> ------------------------------------------------------------------------
> *From:* NANOG <nanog-bounces+martijnschmidt=i3d.net@nanog.org> on behalf
> of Bryan Holloway <bryan@shout.net>
> *Sent:* 19 January 2021 18:18
> *To:* William Herrin <bill@herrin.us>
> *Cc:* NANOG list <nanog@nanog.org>
> *Subject:* Re: Hosting recommendations ... ?
> Perhaps I'm missing something, but in your #1 example "Cloud", what
> prevents me from running a Proxmox ISO (which is more or less Debian)
> vs. a "standard" Debian install on the provider's virtual server?
>
> If I can, I've succeeded. That is the sort of hosting provider I'm
> looking for, if they exist.
>
> #2 would be suitable, but it seems to be that if leased bare-metal dies,
> it will be some time for ETR. Less desirable, but I'm open to ideas.
>
> #3 I do now. Trying to move away from that.
>
>
> On 1/19/21 5:44 PM, William Herrin wrote:
>> On Tue, Jan 19, 2021 at 8:31 AM Bryan Holloway <bryan@shout.net> wrote:
>>> I would like to stop personally dealing with bare-metal. That's what I'm
>>> doing now.
>>
>> Hi Bryan,
>>
>> Cloud = you get virtual servers with virtual storage, generally
>> adjustable to meet your needs. You manage the operating systems and
>> storage within the virtual environment. You DO NOT manage the host
>> operating systems or hypervisors.
>>
>> Bare metal = you lease physical equipment. You manage all software on
>> the equipment including any hypervisors needed to run virtual servers.
>> You DO NOT deal with hardware break/fix, that problem belongs to the
>> service provider.
>>
>> Colocation = You lease space in a data center. You provide physical
>> equipment in your custom configuration.
>>
>> With this terminology, at least one of your requirements is unmeetable
>> for contradicting the others. So I ask again for clarification: which
>> of these do you seek?
>>
>> Regards,
>> Bill Herrin
>>
Re: Hosting recommendations ... ? [ In reply to ]
AceHost.com or Hivelocity.

Sent from my Verizon, Samsung Galaxy smartphone
Get Outlook for Android<https://aka.ms/ghei36>

________________________________
From: NANOG <nanog-bounces+michael=spears.io@nanog.org> on behalf of Bryan Holloway <bryan@shout.net>
Sent: Tuesday, January 19, 2021 12:37:14 PM
To: Martijn Schmidt <martijnschmidt@i3d.net>; William Herrin <bill@herrin.us>
Cc: NANOG list <nanog@nanog.org>
Subject: Re: Hosting recommendations ... ?

You make an excellent point, Martijn ... (and I suspect this is what
Bill was pointing out ...)

Virtualization on top of virtualization is inherently not the best idea.

I guess I'm looking for flexibility in the sense of being able to spin
up additional VMs at my leisure. In which case #2 could be suitable in
the right environment.

... in which case SLAs would be tantamount to success ...

So ... that said, any recommendations? :)


On 1/19/21 6:32 PM, Martijn Schmidt wrote:
> For #1, are you trying to do "Cloud-ception" e.g. running your own
> proxmox virtualization on top of an already virtual machine, so that
> you're basically two layers deep?
>
> For #2, of course you need to be able to survive a hardware failure
> (using RAID1 or some flavour of DRBD for example) but having to think
> about such things is the "trade-off" of having access to the bare-metal
> layer.. it does have advantages, for example if you want to install your
> own virtualization layer without any involvement from the hosting
> provider. You'd usually have agreements with the hosting provider about
> how/when hardware replacements would be done.
>
> Best regards,
> Martijn
> ------------------------------------------------------------------------
> *From:* NANOG <nanog-bounces+martijnschmidt=i3d.net@nanog.org> on behalf
> of Bryan Holloway <bryan@shout.net>
> *Sent:* 19 January 2021 18:18
> *To:* William Herrin <bill@herrin.us>
> *Cc:* NANOG list <nanog@nanog.org>
> *Subject:* Re: Hosting recommendations ... ?
> Perhaps I'm missing something, but in your #1 example "Cloud", what
> prevents me from running a Proxmox ISO (which is more or less Debian)
> vs. a "standard" Debian install on the provider's virtual server?
>
> If I can, I've succeeded. That is the sort of hosting provider I'm
> looking for, if they exist.
>
> #2 would be suitable, but it seems to be that if leased bare-metal dies,
> it will be some time for ETR. Less desirable, but I'm open to ideas.
>
> #3 I do now. Trying to move away from that.
>
>
> On 1/19/21 5:44 PM, William Herrin wrote:
>> On Tue, Jan 19, 2021 at 8:31 AM Bryan Holloway <bryan@shout.net> wrote:
>>> I would like to stop personally dealing with bare-metal. That's what I'm
>>> doing now.
>>
>> Hi Bryan,
>>
>> Cloud = you get virtual servers with virtual storage, generally
>> adjustable to meet your needs. You manage the operating systems and
>> storage within the virtual environment. You DO NOT manage the host
>> operating systems or hypervisors.
>>
>> Bare metal = you lease physical equipment. You manage all software on
>> the equipment including any hypervisors needed to run virtual servers.
>> You DO NOT deal with hardware break/fix, that problem belongs to the
>> service provider.
>>
>> Colocation = You lease space in a data center. You provide physical
>> equipment in your custom configuration.
>>
>> With this terminology, at least one of your requirements is unmeetable
>> for contradicting the others. So I ask again for clarification: which
>> of these do you seek?
>>
>> Regards,
>> Bill Herrin
>>
Re: Hosting recommendations ... ? [ In reply to ]
On 1/19/21 6:33 PM, Brandon Martin wrote:

> On 1/19/21 11:44 AM, William Herrin wrote:

>> Cloud = you get virtual servers with virtual storage, generally
>> adjustable to meet your needs. You manage the operating systems and
>> storage within the virtual environment. You DO NOT manage the host
>> operating systems or hypervisors.

>
> It's worth pointing out that nested virtualization is a thing these
> days, and some providers might even support it!  That means you could
> buy one large instance and sub-divide it yourself into multiple VMs if
> you want to.
>
> In practice, unless you need that flexibility to dynamically spin the
> VMs up and down with various specs AND don't want to or cannot use a
> provider's API for that, I'm not sure why you'd want to if you didn't
> have to for some crazy reason.

I'm very curious about your assertion:

Is nested virtualization really a thing?

I mean, I'm not exactly trying to render Pixar's latest movie ... just
trying to push some bits around (light web-sites, some e-mail ...)

It just seems inherently prone to issues.

Could you back this up with any white-papers or documentation on the
subject? I'm genuinely interested ...

- bryan
Re: Hosting recommendations ... ? [ In reply to ]
On 1/19/21 9:33 AM, Brandon Martin wrote:
> On 1/19/21 11:44 AM, William Herrin wrote:
>> Cloud = you get virtual servers with virtual storage, generally
>> adjustable to meet your needs. You manage the operating systems and
>> storage within the virtual environment. You DO NOT manage the host
>> operating systems or hypervisors.
>
> It's worth pointing out that nested virtualization is a thing these
> days, and some providers might even support it!  That means you could
> buy one large instance and sub-divide it yourself into multiple VMs if
> you want to.
>
> In practice, unless you need that flexibility to dynamically spin the
> VMs up and down with various specs AND don't want to or cannot use a
> provider's API for that, I'm not sure why you'd want to if you didn't
> have to for some crazy reason.


Except for the problem that you pay a premium for larger vm's. you can't
compete with t instances on aws, for starters.

Mike
Re: Hosting recommendations ... ? [ In reply to ]
Maybe this will help:

I use vultr. I have also brought my own address space and am announcing
it to them from one of their instances (vm's) with BGP. They are set up
such that you can use a private AS if you don't have your own and are ok
with them announcing this from their AS (after they strip off the private
as).

If you told me right this second that you needed a Ubuntu Server spun up, I
could be logged into the interface, select a location/size and have it
running within 60 seconds - with the complete Ubuntu install done. Most of
my instances are running under their $5 plan which is 25GB of disk space, 1
CPU, 1G of memory, and 1TB of transfer. Oh, and one or two are on the $6
plan which has slightly more disk and is on a faster CPU. They
obviously have various options that go up from there all the way to
dedicated servers. In a couple of locations, they also have a smaller
IPv6-only server for $2.50.

I haven't found anything I need anything larger than their $5 or $6 plan
for, but I also am not running any heavy workloads. Basically a static
webpage, a ticketing system, a couple of DNS servers, and the like. I'm
currently testing voip(Asterisk) on the platform as well, and based on
testing so far will likely be moving it into full production in the next
few weeks.

I used ubuntu as an example, they also have all of the mainstream linux
distros and the BSD's and I think in some locations windows available.
They also have a pretty good selection of preconfigured applications (aka
common CMS'es, eCommerce platforms, etc). All of these are in the ~60
seconds to spin up category. You can also upload your own ISO or use a
pre-uploaded one from their library of less commonly used
applications/operating systems.

I've been running on the platform for about a year after having enough of a
shared hosting provider for some of the stuff and running on my own metal
in a datacenter for other stuff. So far I'm very happy with them.

If you want to try them out, I noticed as I logged in to look up what the
$5 service includes that they're running a a "refer a friend and they get
$100 to test the service" promo. I normally don't pass on referral codes
but I figured if you're evaluating this you might want the $100 credit (for
up to 30 days of service). If so, the link for that offer is
https://www.vultr.com/?ref=8776996-6G .

On Tue, Jan 19, 2021 at 10:39 AM Bryan Holloway <bryan@shout.net> wrote:

> You make an excellent point, Martijn ... (and I suspect this is what
> Bill was pointing out ...)
>
> Virtualization on top of virtualization is inherently not the best idea.
>
> I guess I'm looking for flexibility in the sense of being able to spin
> up additional VMs at my leisure. In which case #2 could be suitable in
> the right environment.
>
> ... in which case SLAs would be tantamount to success ...
>
> So ... that said, any recommendations? :)
>
>
> On 1/19/21 6:32 PM, Martijn Schmidt wrote:
> > For #1, are you trying to do "Cloud-ception" e.g. running your own
> > proxmox virtualization on top of an already virtual machine, so that
> > you're basically two layers deep?
> >
> > For #2, of course you need to be able to survive a hardware failure
> > (using RAID1 or some flavour of DRBD for example) but having to think
> > about such things is the "trade-off" of having access to the bare-metal
> > layer.. it does have advantages, for example if you want to install your
> > own virtualization layer without any involvement from the hosting
> > provider. You'd usually have agreements with the hosting provider about
> > how/when hardware replacements would be done.
> >
> > Best regards,
> > Martijn
> > ------------------------------------------------------------------------
> > *From:* NANOG <nanog-bounces+martijnschmidt=i3d.net@nanog.org> on
> behalf
> > of Bryan Holloway <bryan@shout.net>
> > *Sent:* 19 January 2021 18:18
> > *To:* William Herrin <bill@herrin.us>
> > *Cc:* NANOG list <nanog@nanog.org>
> > *Subject:* Re: Hosting recommendations ... ?
> > Perhaps I'm missing something, but in your #1 example "Cloud", what
> > prevents me from running a Proxmox ISO (which is more or less Debian)
> > vs. a "standard" Debian install on the provider's virtual server?
> >
> > If I can, I've succeeded. That is the sort of hosting provider I'm
> > looking for, if they exist.
> >
> > #2 would be suitable, but it seems to be that if leased bare-metal dies,
> > it will be some time for ETR. Less desirable, but I'm open to ideas.
> >
> > #3 I do now. Trying to move away from that.
> >
> >
> > On 1/19/21 5:44 PM, William Herrin wrote:
> >> On Tue, Jan 19, 2021 at 8:31 AM Bryan Holloway <bryan@shout.net> wrote:
> >>> I would like to stop personally dealing with bare-metal. That's what
> I'm
> >>> doing now.
> >>
> >> Hi Bryan,
> >>
> >> Cloud = you get virtual servers with virtual storage, generally
> >> adjustable to meet your needs. You manage the operating systems and
> >> storage within the virtual environment. You DO NOT manage the host
> >> operating systems or hypervisors.
> >>
> >> Bare metal = you lease physical equipment. You manage all software on
> >> the equipment including any hypervisors needed to run virtual servers.
> >> You DO NOT deal with hardware break/fix, that problem belongs to the
> >> service provider.
> >>
> >> Colocation = You lease space in a data center. You provide physical
> >> equipment in your custom configuration.
> >>
> >> With this terminology, at least one of your requirements is unmeetable
> >> for contradicting the others. So I ask again for clarification: which
> >> of these do you seek?
> >>
> >> Regards,
> >> Bill Herrin
> >>
>


--
- Forrest
Re: Hosting recommendations ... ? [ In reply to ]
I would look at https://www.webhostingtalk.com forums and specifically the
"Dedicated Hosting Offers" forums
https://www.webhostingtalk.com/forumdisplay.php?f=36 for providers and
deals.

On Tue, Jan 19, 2021 at 1:34 PM Forrest Christian (List Account) <
lists@packetflux.com> wrote:

> Maybe this will help:
>
> I use vultr. I have also brought my own address space and am announcing
> it to them from one of their instances (vm's) with BGP. They are set up
> such that you can use a private AS if you don't have your own and are ok
> with them announcing this from their AS (after they strip off the private
> as).
>
> If you told me right this second that you needed a Ubuntu Server spun up,
> I could be logged into the interface, select a location/size and have it
> running within 60 seconds - with the complete Ubuntu install done. Most of
> my instances are running under their $5 plan which is 25GB of disk space, 1
> CPU, 1G of memory, and 1TB of transfer. Oh, and one or two are on the $6
> plan which has slightly more disk and is on a faster CPU. They
> obviously have various options that go up from there all the way to
> dedicated servers. In a couple of locations, they also have a smaller
> IPv6-only server for $2.50.
>
> I haven't found anything I need anything larger than their $5 or $6 plan
> for, but I also am not running any heavy workloads. Basically a static
> webpage, a ticketing system, a couple of DNS servers, and the like. I'm
> currently testing voip(Asterisk) on the platform as well, and based on
> testing so far will likely be moving it into full production in the next
> few weeks.
>
> I used ubuntu as an example, they also have all of the mainstream linux
> distros and the BSD's and I think in some locations windows available.
> They also have a pretty good selection of preconfigured applications (aka
> common CMS'es, eCommerce platforms, etc). All of these are in the ~60
> seconds to spin up category. You can also upload your own ISO or use a
> pre-uploaded one from their library of less commonly used
> applications/operating systems.
>
> I've been running on the platform for about a year after having enough of
> a shared hosting provider for some of the stuff and running on my own metal
> in a datacenter for other stuff. So far I'm very happy with them.
>
> If you want to try them out, I noticed as I logged in to look up what the
> $5 service includes that they're running a a "refer a friend and they get
> $100 to test the service" promo. I normally don't pass on referral codes
> but I figured if you're evaluating this you might want the $100 credit (for
> up to 30 days of service). If so, the link for that offer is
> https://www.vultr.com/?ref=8776996-6G .
>
> On Tue, Jan 19, 2021 at 10:39 AM Bryan Holloway <bryan@shout.net> wrote:
>
>> You make an excellent point, Martijn ... (and I suspect this is what
>> Bill was pointing out ...)
>>
>> Virtualization on top of virtualization is inherently not the best idea.
>>
>> I guess I'm looking for flexibility in the sense of being able to spin
>> up additional VMs at my leisure. In which case #2 could be suitable in
>> the right environment.
>>
>> ... in which case SLAs would be tantamount to success ...
>>
>> So ... that said, any recommendations? :)
>>
>>
>> On 1/19/21 6:32 PM, Martijn Schmidt wrote:
>> > For #1, are you trying to do "Cloud-ception" e.g. running your own
>> > proxmox virtualization on top of an already virtual machine, so that
>> > you're basically two layers deep?
>> >
>> > For #2, of course you need to be able to survive a hardware failure
>> > (using RAID1 or some flavour of DRBD for example) but having to think
>> > about such things is the "trade-off" of having access to the bare-metal
>> > layer.. it does have advantages, for example if you want to install
>> your
>> > own virtualization layer without any involvement from the hosting
>> > provider. You'd usually have agreements with the hosting provider about
>> > how/when hardware replacements would be done.
>> >
>> > Best regards,
>> > Martijn
>> > ------------------------------------------------------------------------
>> > *From:* NANOG <nanog-bounces+martijnschmidt=i3d.net@nanog.org> on
>> behalf
>> > of Bryan Holloway <bryan@shout.net>
>> > *Sent:* 19 January 2021 18:18
>> > *To:* William Herrin <bill@herrin.us>
>> > *Cc:* NANOG list <nanog@nanog.org>
>> > *Subject:* Re: Hosting recommendations ... ?
>> > Perhaps I'm missing something, but in your #1 example "Cloud", what
>> > prevents me from running a Proxmox ISO (which is more or less Debian)
>> > vs. a "standard" Debian install on the provider's virtual server?
>> >
>> > If I can, I've succeeded. That is the sort of hosting provider I'm
>> > looking for, if they exist.
>> >
>> > #2 would be suitable, but it seems to be that if leased bare-metal dies,
>> > it will be some time for ETR. Less desirable, but I'm open to ideas.
>> >
>> > #3 I do now. Trying to move away from that.
>> >
>> >
>> > On 1/19/21 5:44 PM, William Herrin wrote:
>> >> On Tue, Jan 19, 2021 at 8:31 AM Bryan Holloway <bryan@shout.net>
>> wrote:
>> >>> I would like to stop personally dealing with bare-metal. That's what
>> I'm
>> >>> doing now.
>> >>
>> >> Hi Bryan,
>> >>
>> >> Cloud = you get virtual servers with virtual storage, generally
>> >> adjustable to meet your needs. You manage the operating systems and
>> >> storage within the virtual environment. You DO NOT manage the host
>> >> operating systems or hypervisors.
>> >>
>> >> Bare metal = you lease physical equipment. You manage all software on
>> >> the equipment including any hypervisors needed to run virtual servers.
>> >> You DO NOT deal with hardware break/fix, that problem belongs to the
>> >> service provider.
>> >>
>> >> Colocation = You lease space in a data center. You provide physical
>> >> equipment in your custom configuration.
>> >>
>> >> With this terminology, at least one of your requirements is unmeetable
>> >> for contradicting the others. So I ask again for clarification: which
>> >> of these do you seek?
>> >>
>> >> Regards,
>> >> Bill Herrin
>> >>
>>
>
>
> --
> - Forrest
>
Re: Hosting recommendations ... ? [ In reply to ]
On Tue, Jan 19, 2021 at 9:18 AM Bryan Holloway <bryan@shout.net> wrote:
> Perhaps I'm missing something, but in your #1 example "Cloud", what
> prevents me from running a Proxmox ISO (which is more or less Debian)
> vs. a "standard" Debian install on the provider's virtual server?

Hi Bryan,

I haven't used Proxmox but from a 60 second glance through Google that
looks like you're asking for nested virtualization. If it works at
all, you'd take a double-hit on everything that wants to run in ring
0, a double-hit on virtualized I/O and a double-hit for OS overhead
making the result more than a little sluggish. Kinda has "bad idea"
written all over it.

As I understand it, you can "cat /sys/module/kvm*/parameters/nested"
in one of the service provider's VMs and if the answer is "1" or "Y"
for the CPU type which matches the exposed CPU then what you're asking
for will probably work. For some definition of work anyway.


I use Vultr for my primary BGP exit and have found it largely
painless. The VMs I have there DO NOT support nested virtualization.
They do claim a bare metal offering but it's currently listed as sold
out in all of their data centers. They also claim to provide mountable
block storage for compute instances up to 10TB per, but I haven't
worked with that feature, I presume it only applies to virtual
servers, and it looks like it's only available in one of their data
centers.

Regards,
Bill Herrin



--
Hire me! https://bill.herrin.us/resume/
Re: Hosting recommendations ... ? [ In reply to ]
Proxmox specifically isn't much more than a wrapper for standard Linux KVM,
which can support nested virtualization. In my limited experience with
nesting, it doesn't work half bad as one would expect, but I haven't used
it in a stressed environment with anything substantial running that way.


On Tue, Jan 19, 2021 at 1:53 PM William Herrin <bill@herrin.us> wrote:

> On Tue, Jan 19, 2021 at 9:18 AM Bryan Holloway <bryan@shout.net> wrote:
> > Perhaps I'm missing something, but in your #1 example "Cloud", what
> > prevents me from running a Proxmox ISO (which is more or less Debian)
> > vs. a "standard" Debian install on the provider's virtual server?
>
> Hi Bryan,
>
> I haven't used Proxmox but from a 60 second glance through Google that
> looks like you're asking for nested virtualization. If it works at
> all, you'd take a double-hit on everything that wants to run in ring
> 0, a double-hit on virtualized I/O and a double-hit for OS overhead
> making the result more than a little sluggish. Kinda has "bad idea"
> written all over it.
>
> As I understand it, you can "cat /sys/module/kvm*/parameters/nested"
> in one of the service provider's VMs and if the answer is "1" or "Y"
> for the CPU type which matches the exposed CPU then what you're asking
> for will probably work. For some definition of work anyway.
>
>
> I use Vultr for my primary BGP exit and have found it largely
> painless. The VMs I have there DO NOT support nested virtualization.
> They do claim a bare metal offering but it's currently listed as sold
> out in all of their data centers. They also claim to provide mountable
> block storage for compute instances up to 10TB per, but I haven't
> worked with that feature, I presume it only applies to virtual
> servers, and it looks like it's only available in one of their data
> centers.
>
> Regards,
> Bill Herrin
>
>
>
> --
> Hire me! https://bill.herrin.us/resume/
>
Re: Hosting recommendations ... ? [ In reply to ]
On 1/19/21 1:50 PM, William Herrin wrote:
> I haven't used Proxmox but from a 60 second glance through Google that
> looks like you're asking for nested virtualization. If it works at
> all, you'd take a double-hit on everything that wants to run in ring
> 0, a double-hit on virtualized I/O and a double-hit for OS overhead
> making the result more than a little sluggish. Kinda has "bad idea"
> written all over it.

KVM, at least, and I think Xen as well, have some features for
"shunting" I/O and hypervisor calls through to the bare-metal hypervisor
where possible and avoiding double processing and trampolining. It's
not nearly as bad as you might think in terms of performance as long as
the hardware supports it (nested page tables being the big one). The
little I've played with it mostly has proven to be an administrative
hassle rather than performance.

I would not recommend mixing and matching hypervisors (e.g. Xen on KVM
or vice-versa), though. I'm not even sure you can do so meaningfully,
though I bet someone's working on it.
--
Brandon Martin
Re: Hosting recommendations ... ? [ In reply to ]
On Tue, Jan 19, 2021 at 10:50:31AM -0800, William Herrin wrote:
> I use Vultr for my primary BGP exit and have found it largely
> painless. The VMs I have there DO NOT support nested virtualization.

That's odd, I have run Xen in standard Vultr VMs, and it works. It's not
fast, though.

Thanks,
Ward.
Re: Hosting recommendations ... ? [ In reply to ]
On 1/19/21 12:56 PM, Bryan Holloway wrote:
>
> I'm very curious about your assertion:
>
> Is nested virtualization really a thing?
>
> I mean, I'm not exactly trying to render Pixar's latest movie ... just
> trying to push some bits around (light web-sites, some e-mail ...)
>
> It just seems inherently prone to issues.
>
> Could you back this up with any white-papers or documentation on the
> subject? I'm genuinely interested ...

With KVM, if you have a recent kernel and qemu, it pretty much "just
works" on supported hardware. AFAIK Xen supports "Xen on Xen", too, but
I haven't used it and don't know much about it.

The use case is pretty much exactly this. You (the product consumer)
are handed a product that amounts to a virtual machine on somebody
else's $BIGBOX. You want to deploy multiple virtual machines where you
have direct control over their lifecycle, configuration, etc. and can
bring in additional I/O resources, etc. at the hypervisor level
(consider that, with KVM, the Linux kernel basically IS the hypervisor).
So, you run one or more VMs inside the top level VM that you're handed.

It's full of lots of little wiggles and can be a pain to maintain if you
have visibility into both levels of the equation, but it does seem to
work and is surprisingly performant.

See e.g. https://tips.graphica.com.au/nested-kvm/
--
Brandon Martin
Re: Hosting recommendations ... ? [ In reply to ]
On Tue, Jan 19, 2021 at 3:27 PM Brandon Martin <lists.nanog@monmotha.net>
wrote:

> On 1/19/21 12:56 PM, Bryan Holloway wrote:
> >
> > I'm very curious about your assertion:
> >
> > Is nested virtualization really a thing?
> >
> > I mean, I'm not exactly trying to render Pixar's latest movie ... just
> > trying to push some bits around (light web-sites, some e-mail ...)
> >
> > It just seems inherently prone to issues.
> >
> > Could you back this up with any white-papers or documentation on the
> > subject? I'm genuinely interested ...
>
> With KVM, if you have a recent kernel and qemu, it pretty much "just
> works" on supported hardware. AFAIK Xen supports "Xen on Xen", too, but
> I haven't used it and don't know much about it.
>
> The use case is pretty much exactly this. You (the product consumer)
> are handed a product that amounts to a virtual machine on somebody
> else's $BIGBOX. You want to deploy multiple virtual machines where you
> have direct control over their lifecycle, configuration, etc. and can
> bring in additional I/O resources, etc. at the hypervisor level
> (consider that, with KVM, the Linux kernel basically IS the hypervisor).
> So, you run one or more VMs inside the top level VM that you're handed.
>
> It's full of lots of little wiggles and can be a pain to maintain if you
> have visibility into both levels of the equation, but it does seem to
> work and is surprisingly performant.
>

Nested virtualization is, however, pretty widely deployed for production
workloads. Take for instance Juniper's vSRX and vMX products. Both require
nested virtualization support as they run a Wind River Linux instance with
KVM directly on your virtual machine which may be on vmware, KVM, Hyper-V,
etc, and then Junos running nested within the WRL KVM virtual machine. So
it's not like this is something that's uncommon or not trustworthy, lots of
us are doing it every day with no serious problems and entirely acceptable
performance on modern hardware. I run a decent fleet of vSRX's on Hyper-V
and it works well. YMMV as always based on your own platform, but I don't
think that nested virtualization is something that we should be steering
clear of at this point.

- mdh

Matt Harris|Infrastructure Lead Engineer
816-256-5446|Direct
Looking for something?
Helpdesk Portal|Email Support|Billing Portal
We build and deliver end-to-end IT solutions.
RE: Hosting recommendations ... ? [ In reply to ]
>Is nested virtualization really a thing?

Real Computers have been running VMs inside VMs for about 50 years. Bringing this technology to "bitty boxes" is a recent thing.

--
Be decisive. Make a decision, right or wrong. The road of life is paved with flat squirrels who could not make a decision.
Re: Hosting recommendations ... ? [ In reply to ]
On 1/19/21 10:56 AM, Bryan Holloway wrote:
> On 1/19/21 6:33 PM, Brandon Martin wrote:
>> On 1/19/21 11:44 AM, William Herrin wrote:
>>> Cloud = you get virtual servers with virtual storage, generally
>>> adjustable to meet your needs. You manage the operating systems and
>>> storage within the virtual environment. You DO NOT manage the host
>>> operating systems or hypervisors.
>> It's worth pointing out that nested virtualization is a thing these
>> days, and some providers might even support it!  That means you could
>> buy one large instance and sub-divide it yourself into multiple VMs
>> if you want to.
>>
>> In practice, unless you need that flexibility to dynamically spin the
>> VMs up and down with various specs AND don't want to or cannot use a
>> provider's API for that, I'm not sure why you'd want to if you didn't
>> have to for some crazy reason.
>
> I mean, I'm not exactly trying to render Pixar's latest movie ... just
> trying to push some bits around (light web-sites, some e-mail ...)
There is actually a happy medium.  Spin up a large VM.  Install
Proxmox.  But instead of spinning up heavy duty VM in KVM, or such,
Proxmox knows how to spin up LXC containers.  You get light performant
segregation between applications without the duplication of kernel.

Of course, if you need to run some sort of windows thingy that WinHQ
can't solve, then you have the ability to spin up a 'heavy' VM.

For native Linux apps, as I know how to do container networking
directly, many times I just skip the ProxMox abstraction and run the LXC
commands and IP commands directly.

Raymond Burkholder
blog: https://blog.raymond.burkholder.net

1 2  View All