Update on Basecamp 3 being stuck in read-only as of Nov 8, 9:22am CST

Basecamp 3 is now back online for reading and writing. All data was confirmed to be fully safe and intact. No emails that were sent to Basecamp during the outage were dropped. We may still have some backlogs on processing things like incoming emails, and you may still see some slowdowns here and there as we catch up. But we are back, and we are safe.

We will be following up with a detailed and complete postmortem soon. All in, we were stuck in read-only mode for almost five hours. That’s the most catastrophic failure we’ve had a Basecamp in maybe as much as decade, and we could not be more sorry. We know that Basecamp customers depend on being able to get to their data and carry on the work, and today we failed you on that.

We’ve let you down on an avoidable issue that we should have been on top of. We will work hard to regain your trust, and to get back to our normal, boring schedule of 99.998% uptime.

Note: If you were in the middle of posting something new to Basecamp, and you got an error, that data is most likely saved in our browser-based autosave system. If it doesn’t appear automatically, we can help you recover that data. Please contact support if you’re in this situation, and we’ll have a team ready to assist.

Below is the timeline for today:

At 7:21am CST, we first got alerted that we had run out of ID numbers on an important tracking table in the database. This was because the column in database was configured as an integer rather than a big integer. The integer runs out of numbers at 2147483647. The big integer can grow until 9223372036854775807.

At 7:29am CST, the team diagnosed the problem and started working on the fix. This meant writing what’s called a database migration where you change the column type from the regular integer to the big integer type. Changing a production database is serious business, so we had to test this fix on a staging database to make sure it was safe.

At 7:52am CST, we had verified that the fix was correct and tested it on a staging database, so we commenced making the change to the production database table. That table in the database is very large, of course. That’s why it ran out of regular integers. So the migration was estimated to take about one hour and forty minutes.

At 10:56am CST, we completed the upgrade to the databases. This was the largest part of the fix we needed to address the problem. But we still have to verify all the data, update our configurations, and ensure that we won’t have more problems when we go back online. We’re working on this as fast as we can.

At 11:33am CST, we’re still verifying that all data is as it should be for Basecamp 3. The database migration has finished, but the verification process is still ongoing. We’re working as fast as we can and hope to be back fully shortly.

At 11:52am CST, verification of the databases is taking longer than expected. We have 4 databases per datacenter and we have two datacenters with databases. So a total of 8 databases. We need to be absolutely certain that all the data is in proper sync before we can go back online. It’s looking good, but 99% sure isn’t good enough. Need 100%.

At 12:22pm CST, Basecamp came back online after we successfully verified that all data was 100% intact.

At 12:33pm CST, Basecamp had another issue dealing with the intense load of the application being back online. This caused a caching server to get overwhelmed. So Basecamp is down again while we get this sorted.

At 12:41pm CST, Basecamp came back online after we switched over to our backup caching servers. Everything is working as of this moment, but we’re obviously not entirely out of the woods yet. We remain on red alert.

I will continue to update this post with more information, and we will provide a full postmortem after this has completed.


Further insight on the technical problem: It’s embarrassing to admit, but the root cause of this issue with running out of integers has been a known problem in our technical community. We use the development framework Rails (which we created!), and the default setting for that framework move from integer to big integer two years ago.

We should have known better. We should have done our due diligence when this improvement was made to the framework two years ago. I accept full responsibility for failing to heed that warning, and by extension for causing the multi-hour outage today. I’m really, really sorry 😢

What’s new in Basecamp 3.6 on iOS

This feature-packed release of Basecamp for iPhone and iPad is available in the App Store today. Here’s a look at what’s new.

Improved attachments and sketching

It all starts with a redesigned file picker. Tap the paperclip button anywhere in Basecamp to see clear buttons for each kind of thing you can attach. They’re all first-class — especially Sketch which got a big boost in this release. Now, before you upload an image to Basecamp you’ll have the option to draw on it first. It’s great for highlighting and making notes — or just having fun.

Pick an image (left), tap ‘Sketch on image’, then add your drawings before uploading to Basecamp.

In addition to sketching on images, we’ve also beefed-up the drawing tools. You can now choose the from 3 line weights and 5 colors to add variety and interest to your sketches. Also new: save your Basecamp sketches or share them to other apps.

Works great with Apple Pencil on iPad Pro, too.

Drag and Drop Files on iPad

One of the coolest new features on iOS 11 is drag and drop and it’s now supported in Basecamp. You can now select one (or more) images from the Photos app, for example, and simply drop them into Basecamp! Here’s how it looks:

Drag one or multiple files into Basecamp.

Easier invites

Awhile back, inviting people to your projects got easier with the introduction of special links you could send to people that would automatically invite them to the project — no need to enter their name and email. On iOS we took that a step further. With one tap you can now share the URL with others via Messages, Email, Airdrop — or any other apps you use on iOS. It’s the easiest way to get people into your projects yet!


iOS 11 updates

Finally this release includes several fixes and improvements for iOS 11. The most notable one is for people who were unable to upload images to Basecamp because they were using iOS 11’s new space-saving HEIC format. Now when you upload an HEIC image, Basecamp will automatically convert it to a compatible format (jpg). It all happens automatically and behind-the-scenes so you won’t have to do a thing—it just works!

That’s all for now. We’re cooking up more for the next release. Stay tuned!

As always, please keep your suggestions, feedback, and bug reports coming our way. We’ve got some neat stuff coming in the next version so if you’re interesting in seeing it before everyone else, we have a few openings in our private beta. Send us an email and we’ll invite you.

❤️ The iOS Team at Basecamp, 
Tara Mann, Dylan Ginsburg, Zach Waugh, and me.

Basecamp 3 for Android, a big update


The Android Team has been on a roll, steadily improving the Basecamp 3 app these past few months. We’ve even added some new features specifically for Android 8.0 Oreo devices. Here’s a summary of all the good stuff we’ve added since summer.

  • Vastly improved image, media, and document handling
  • Easier to share links, text, images, and more from other apps and across Basecamp
  • Automatic links to invite people to your Projects and Teams
  • The ability to sign-up for a new account with a 30-day free trial

Introducing the new File Browser.

Gallery View

A brand new File Browser shows you a gallery view of the last 100 images, videos, and GIFs stored on your device. Tap a thumbnail in the gallery to attach a file to a comment, check-in answer, Campfire, or Ping.

Tap the attachment icon (paperclip) to open the new File Browser.

Sketch Tool

With the Sketch tool you can draw on and annotate images. This is particularly handy if you need to add notes to a screenshot. You can even give the file a more descriptive name before uploading the drawing to Basecamp.

Sketch on any photo you’re about to upload to Basecamp.

The Sketch tool can also be used to create a standalone drawing. Draw a diagram using up to 4 colors. Make a mistake? No worries, you can undo multiple times. A copy of your sketch is also saved in your device’s photo gallery.

Sketch out an idea to share with your team.

Camera and Video Tools

You can now directly access your device’s built-in camera from the app. Capture your surroundings with the Camera tool. Or record an update for your team with the Video tool.

Direct access to your device’s camera from the app.

Browse Tool

The Browse tool integrates with your device’s file browser. You can access PDFs from Dropbox, spreadsheets from Google Drive, photos from Google Photos, etc. Your preferred cloud storage service will be at your fingertips.

Browse Dropbox, Google Drive, or your preferred cloud storage service for files to upload to Basecamp.

Better navigation.

Look back at a discussion

Now you can look back at the discussion as you’re adding a comment. Start writing—flip back to reference the original Message—then continue writing.

Reference the message you’re commenting on.

My Schedule

See an overview of your upcoming events in My Schedule. This lists all the events you’re involved in across all Projects and Teams in Basecamp.

See everything coming up on My Schedule.

Invite your team with a text

Get people working with you in Basecamp by sharing an invite via Messenger, WhatsApp, Email, etc.

Add people to your Project by sharing a link.

Post to a Ping

Send files, YouTube videos, Tweets, websites and more directly to your Ping conversations.

Share links, images, text, and more directly to a Ping.

Improved media file handling.

Inline PDF viewer

Tired of downloading PDFs then searching for them in your Downloads folder just to read them? Now you can open and read your PDFs directly inside of Basecamp. Pan, zoom, and swipe through multi-page documents.

Note: password-protected PDFs must be downloaded to preserve security.

Read PDFs in Basecamp without having to download them to your device.

New audio file support

Previously our Media player only supported videos. You had to download audio files to your device to listen to them. Now you can listen to audio files such as MP3s, AIFs, and WAVs directly in the Media player.

GIFs in Campfires and Pings

Hooray! We added GIF browsing and searching to Campfires and Pings! A favorite feature of ours: perfect for those times when a reaction GIF is needed.

Probably the best feature we’ve made: GIF browser and search in Campfires and Pings.

Additionally you can now change your profile picture and company logo (admins only) in the app.


Updated for Android O.

Support for Android 8.0 Oreo is complete with under the hood optimizations, a new adaptive icon, and finer control over Basecamp notifications. Turn off specific notifications, and set custom sounds for others. Lucky to have Android O on your device? Long-press the app icon to access the Quick Add menu. This new notification control is in App Settings (the “i” icon).

Android O updates include a new adaptive icon, notification dots, and better control over Basecamp notifications.

No Basecamp? No worries.

Create a new Basecamp account right from the login screen

Sign up for a 30-day free trial right after you install the app. Setting up a new company account on Android also gives your team access to Basecamp on the web, iOS, as well as on Mac and Windows computers.

Get the Basecamp 3 for Android update today on Google Play (Version 3.6.6, updated September 27, 2017)


Thanks for reading. If you have any questions about Basecamp 3 for Android please let us know.

— Brought to you by the Android Team at Basecamp: Jamie, Dan, and Jay

Basecamp 3 for iOS: Hybrid Architecture

We’ve written quite a bit in the past about our approach to building hybrid mobile apps. Basecamp 3 represents the latest generation of this architecture, taking everything we’ve learned from previous versions.

The first app for Basecamp 2 app was iPhone only, written in RubyMotion as a thin wrapper around UIWebView. Next, we did a new universal app for Basecamp 2, written in Xcode + Objective-C, still a using UIWebView, but with a bit more native code thrown in. For Basecamp 3, we’ve replaced Objective-C with Swift, UIWebView with WKWebView and added Turbolinks, with even more native code, and a deeper integration between native and web.

Defining Hybrid

First, it helps to be clear about what we mean by “hybrid”. That term is used in so many different contexts, that it’s almost meaningless. In our use, we’re referring to standard native apps where a significant portion of the content is rendered using web technology. I explicitly say content there because it is an important distinction. We’re not using a framework that attempts to mimic native controls using HTML/CSS. We’re not using a framework that tries to compile another language to native code, or make a cross-platform app from a single codebase.

For us, it means using Xcode + Swift, and conforming to all the platforms conventions regarding navigation/presentation. The building blocks of our app are composed of UINavigationController, UITabViewController, UISplitViewController, UIViewController, etc. Within those containers, we have many screens where the content is built using UITableView or UICollectionView, we have even more where that role is filled by a WKWebView.

Under the hood

Basecamp 3 for iOS is written 100% in Swift 3.1 (soon to be 4), using the latest version of Xcode. We only have a few dependencies, but the ones we do have we manage with Carthage. The core library for enabling this hybrid architecture is Turbolinks. We use Turbolinks on the web, and our companion frameworks for iOS and Android let us use it in our native apps as well. The framework handles communicating with Turbolinks.js and allowing the use of a single WKWebView, shared across multiple different screens.

Router/Navigator
In addition to Turbolinks, we have a number of other components to support it. Most of our navigation in the iOS app is URL-driven. A url can come from a number of sources (web link, push notification, universal link from another app, native action, etc), and they all go through the Router. This router is responsible for figuring out exactly what action to take for a given url. The router may open the url in Safari if it’s for another domain, display a media viewer if it’s an image/video, or in the common case, create a new view controller to display. The router hands off a view controller off to the Navigator which handles the presentation. Most view controllers are pushed on the current navigation stack, but we also support presenting certain screens (like new/edit views) modally, or replacing the current screen when appropriate.

Bridge
The last component that makes up the hybrid architecture (though we have a number of other components not related to the hybrid part) is what we call the “Bridge”. This is a umbrella term for all the various parts of the app involved in native→web (or web→native) communication. The primary code here is a local JavaScript file (written in TypeScript) embedded in the app and injected into the web view using WKUserScript. This provides native code an API for communicating with the web view without needing to directly query the DOM or do complex JS. Using a WKScriptMessageHandler, we can respond to messages sent from the web view through the bridge.

Bridge in action. The mobile web is on the left, the app is on the right. The bridge hides the top nav, breadcrumbs, and other elements we want render differently in the app

Above is one example of the bridge in action. We use it to hide many elements that are normally displayed on the mobile web that don’t make sense in the app. Since we provide a tab bar for top-level navigation, we don’t need that displayed here. Since we have a navigation controller, we don’t need the breadcrumbs for navigation. Finally, we hide the web edit/bookmark/actions menu and instead provide a native version.

Examples

This is easier to visualize what this looks like in practice with a few examples. In the images below, I’ll use a purple overlay to indicate web view, and a green overlay to indicate native UI.

Main Tabs
Basecamp 3 for iOS has 4 main tabs (Home, Hey!, Activity, and Find). Each one of these tabs are 100% native. These are the primary points of interaction in the app, and we wanted them to be as fast as possible. We also wanted to provide a different experience from the desktop that we thought made more sense on mobile, such as a unified Hey! for all notifications that also included recent Pings.

Basecamp 3 for iOS — Main tabs

Message
When you tap a notification in Hey!, say for a new message, then we push a new TurbolinksViewController on the navigation stack:

From left to right: Hey! screen, viewing a message, actions menu, tools menu

This is a typical screen where all the content is a web view. Through our bridge, we pulled data out of the page to display in the navigation bar. Similarly, we used data from the DOM to populate a native actions menu popover displayed when you tap the “…” button. Since this dynamic and provided by the page, we can change it server-side at any time. Finally, if you tap the nav bar title, we show a native “tools menu” that provides quick access for navigating around a project.

Campfire
We also have screens where the content is a mix of both native and web. This is the case for Campfires:

From left to right: Hey! screen, viewing a campfire, completing a mention, attaching a file

The main chat content here is web, but we decided to use a native view for the input. This fixes a number of issues with the web input like maintaining the correct position when scrolling, and we can also have better control over things like interactive keyboard dismissal. When typing someone’s name, we use a native mention auto-completer. Tapping the paperclip button shows the attachment picker, which is a native element that we use throughout the app with some nice touches, like quickly picking recently taken photos. All these components can work seamlessly together on the same screen.

Summary

Those are just a few examples, but demonstrates the flexibility of this approach. The key to this architecture is that we’re not locked into one method or framework. Native or web isn’t a binary choice, but instead a spectrum:

Web → Native spectrum

For each screen of the app, we can adjust where we sit on that spectrum. We can decide a native screen gets little use and isn’t worth the maintenance, so we change it to web. We can decide a web screen isn’t providing the best experience, and convert it to native. We could decide to try React Native and mix that in as well. Whenever Apple releases a new API, we can immediately support it since we’re not depending on a 3rd-party framework to be updated.

One thing we deeply value at Basecamp is independence of teams. If we had to coordinate the development and release of every feature with the web, iOS, and Android teams all working in lockstep, we’d never ship anything. This architecture allows our web team to build a new feature and ship it simultaneously across all platforms. By default, we have a view controller that can display any url in Basecamp 3, so any new urls will just work in the app. We can iterate and experiment on the web, ship immediately on all platforms, and later make it native if feel we can improve the experience.

This also lets us the mobile teams focus on how to best serve the platform. One of our goals is 100% coverage in the mobile apps, you should never have to go to the desktop because the apps don’t support something. With our solid foundation provided by the web, we can deliver on that goal, and then focus our efforts on platform-specific improvements. These include features like rich content push notifications, universal links, hand-off support, iCloud Keychain support, share extension, today widget, and more. Some of those things would be impossible or non-trivial if we didn’t have full native support at our disposal.

New file upload and browsing in Basecamp 3 for Android

Summer brings 4 day work weeks at Basecamp, but that doesn’t mean the Android team takes a break. They just launched a new version of Basecamp 3 for Android, with an updated file browser to make attaching files to your Basecamp faster and easier.

With the new file browser, you can attach an image directly from the camera (Android 5 and above), or file from Dropbox or Google Drive:

Upload from your camera, or a service like Dropbox/Google Drive.

Check out the new camera and file picking features in action:

Get it Today!

Basecamp 3 for Android 3.5.4 with the new file browsing and uploading is available today in the Google Play Store. The new file browsing feature is available in Android 4.4 and above (the camera option is in Android 5.0 and above only). Try it out and leave Jay, Dan and Jamie a review in the Play Store.

If you don’t have a Basecamp 3 account yet, now is the perfect time to get one. Your first 30 days are free!

New in Basecamp 3 for iOS 3.4.1

You know that with the Android app getting updated so recently, that an update to the iOS app was not far behind. In fact, the iOS team (Jason Z, Tara, Dylan and Zach) launched the latest version last week! It’s got a sweet set of new features I’m excited to share with you.

Hey! Who Moved My Pings?

In previous versions of the app, Pings were a little harder to find and challenging to start. Now Pings are smartly located in the Hey section, right at the top. You’ll see a row of avatars for your most recent pings. You can quickly start a new one or swipe through previous Pings.


You can also quick swipe on items in the Hey menu to mark them as read:


Docs and Files List View

The team also added a list view for Docs & Files, with new file icons, smoother re-ordering, tap to preview images, and swipe to move and archive.

These updates, along with a batch of the usual speed enhancement fixes, have made the iOS app better than ever! You can get the latest version of Basecamp for iPhone and iPad on the App store. If you like it, please leave a review! If you don’t have Basecamp 3 yet, get started with a 30 day free trial now.

Basecamp 3 for Android 3.5 — Catch Up!

When you’ve been away from your Basecamp 3 account for a stretch of time and need to get current, it can be tough to know where to start. On top of that, opening each Hey! item individually can take a while.

That’s why the Android team (Jay, Jamie, and Dan) developed the new Catch Up feature, available in the latest update to the Android app.

Android Catch Up — part of a balanced diet.

How it works

When you have two or more unread Hey! items in your Inbox, you’ll see a “Catch Up” button.


Tap that button, you’ll go to Catch Up mode, where you can swipe through your unread items one by one. The unread count will appear at the top and your items are optimized for easy reading.


My favorite part is using “Mark as Unread” to keep items in the Hey! inbox. This is perfect for when you want to quickly browse through everything, but give a second look important things later.

Here’s what Catch Up looks like in the app, start to finish:

Available now!

Basecamp 3 for Android 3.5 is available from the Google Play Store. It’s got the new Catch Up feature (Android 5.0 and above only) along with the usual group of bug fixes and overall polish. Try it out and leave us a review in the Play Store.

Don’t have Basecamp yet? Sign up today, the first 30 days are free.

New in Basecamp 3: List view for Projects and Teams

The change from Winter to Spring can provide a lot of inspiration for our team. We take time to listen to the rain, the wind, the flowers until they whisper the secrets to the way forward. Frequently, this growing season plants in us the ideas for the future of Basecamp.

Nature can be very gentle, but direct.

OK, OK, the truth of our new List view feature has a lot less to do with communing with nature and a lot more to do with listening to customers, but it’s still exciting! While a lot of customers like the card view, projects with longer titles are truncated and sometimes look strange. With a large number of teams and projects, the card view can be overwhelming.

Designer Scott and programmer Pratik heard the call! They teamed up to create a new feature which available to all Basecamp 3 users today. Now, you can see your teams in projects in a new List view, which groups projects alphabetically for quick scanning:

List view in Basecamp 3

Changing your View

To change your Team or Project, click the ••• menu at the top right of the section.

Switch using the ••• menu at the top right

Jump to your projects and pin them

The Jump menu still works the same with the Card or List view.

Filter your projects with the jump menu

Pinning is available from the ••• button at the right of a project or team:

Pin a project

New Project Flow

With the addition of the list view, we adjusted the flow for creating a new project or team. There’s a cleaner New button, where you can create a new project from scratch or a template and manage your templates.

Add a project

Faster, List View, Load, Load!

Previously, Basecamp 3 would load full menus for pinning and renaming for each project. By loading those menus asynchronously, we reduced HTML on the Home page by 45%. In addition to being just a bit more helpful, this redesign makes loading the whole home page faster.

A GIF of Basecamp loading wouldn’t make sense, so pretend that this is Basecamp.

Try it today!

List view for Teams and Projects act independently of each other. Basecamp 3 will also remember your preference no matter if you’re using the desktop apps or a browser. (List view for mobile apps will come in future updates to those apps.)

A list view was one of our most popular customer requests and we’re very happy to add it in Basecamp 3. If you don’t have a Basecamp 3 account yet, sign up today, the first 30 days are free.

New in Basecamp for Android 3.4

The last update for their app was only a month ago, but Team Android at Basecamp has kept a steady pace to release a new version of the app, chock full of slick new screens and updates to features you already love. Here’s what you’ll find in the latest version, available today:

New Hey! Menu

The new Hey! menu brings all your new @mentions, Messages, Comments, To-dos, Events, and Campfire chats into one place. If it’s new in your account, it’s in your Hey! menu.

Hey! is for horses and Basecamp notifications.

Improved Pings

Pings are a great way to reach out to someone on your account without creating noise for your other colleagues in a Team or Project Campfire. This update makes them even easier to use on Android. Now, each Ping with an individual user gets its own color with a profile picture:

Ping a friend.

Group pings are now clearer. Tap the colored toolbar to see who is participating in the Ping. You can also tap a participant’s profile picture to see that person’s time zone and other information:

Purple is the color of our true group Pings, in the morning when we rise.

You can also send a Ping from any screen in the app. Here’s how it works:

Fixes and Fine Tuning

In addition to those great features above, the team added a little polish all over like swiping right to go back to the previous screen and displaying to-do list descriptions. They also made some adjustments to make drafting and saving messages and comments smoother.

Get it today!

Basecamp 3 for Android 3.4 is available now from the Google Play Store. If you don’t have a Basecamp 3 account yet, sign up today, the first 30 days are free.

Try it out and let us know what you think in a comment below — Jay, Jamie, and Dan would love to hear from you. And if you have a chance, please do leave us a Play Store review too. Thanks!