To use Google Drive you must pass Google's Tier 2 Security Assessment

[cognettings]

Has anyone gone through the process of approving their app with Google? How did it go, what problems did you run into and how did you solve them?

In order to let my users upload files from their Google Drive, I’m required to go through Google’s approval process since I’m self-hosting Companion. I scanned Companion for vulnerabilities and the scanner reported over 200 issues (most of them in dev dependencies). In order to pass I need to resolve all issues reported by the scanner and submit the results to the third-party. Then I need to answer questions about the internals of Companion/Uppy during the self-assessment.

Am I understanding this correctly?

Here’s a link to where I first learned about the app verification requirement: GitHub issue

Below is part of the email I received from Google after submitting my app for approval.

---

Thank you for your patience while we reviewed your project’s use of OAuth restricted scopes.

We have completed our initial review of your app { / }****.

For final approval, you are required to complete a Tier 2 verified self security assessment and be issued a Letter of Validation for your application by your due date September 20 2023. This assessment is required annually and applies to all apps requesting restricted scopes; to learn more, please visit the CASA website: .

The due date is to complete your assessment and receive a Letter of Validation. It can take up to 6 weeks to complete the CASA assessment, so it is important to initiate your assessment as early as possible.

Next Steps

You have the following options to complete your assessment:

1 - Tier 2 Self Scan Using Open Source Tools

• Follow the CASA Tier 2 procedures to self scan your application
• Fix any CWEs flagged by your scan
• Register: or log-in: to the CASA portal and initiate your security assessment
• Submit your scan results and fill out the CASA questionnaire on the portal
• Receive the results and validation report in the CASA portal
• The CASA portal will automatically share the Letter of Validation with Google.

---

The next portion is from an email I received from CASA.

---

Welcome to CASA Tier 2 Assessment Portal. In order to submit your application assessment, please use this link to log in, complete the questionnaire, and upload the scan results document. Our assessment process is as follows:

• Survey Submission and Initial Review:
  • This is an initial high-level review of your submission to validate that the questionnaire has been fully completed and all necessary evidence has been submitted.
  • If your submission is complete, your submission will be progressed to the detailed review phase.
  • If your submission is deemed incomplete, an assessor will be in touch to request the missing details before your review is progressed.
• Detailed Review:
  • This is a detailed review of the assessment questionnaire and related supporting evidence.
  • If everything has been appropriately submitted and your submission meets all the CASA requirements, your submission will be progressed to the final QA phase (where your LOA will be issued).
  • If you have not met all the CASA requirements or your submission is unclear, an assessor will reach out to you to obtain clarification and to resolve any open questions. In this instance, the progression of the detailed review phase will primarily be driven by the time it takes you to resolve the assessor’s questions. In this case, we will still work closely with you to progress your assessment before its due date.
• Final QA and Completion:
  • This is the final QA review of your assessment. If your submission passes the CASA assessment, your LOA will also be issued during this time.

Thank you for your prompt attention to this request.

Sincerely,
The CASA Tier 2 Assessment Team

[dschmidt]

Unfortunately I have no experiences to share, but we will have to go through this very soon. Keep us posted how it goes, please!

[cognettings]

Ok so here’s what I’ve learned so far. You don’t need to go through the assessment to use Google Drive. However, if you want to customize the oauth screen, then you probably have to (I’m still waiting to hear back from Google and CASA).

To access Google Drive files, just use Transloadit’s hosted Companion server (it’s free as long as you don’t apply any processing to the files). However you won’t be able to customize Google’s oauth screen to use the name of your application.

I mainly want to customize the oauth screen so my users see my app’s name when giving consent to access their Google Drive files. I think it’d be pretty odd if I was using an app called “Video Editor’s Generic App Name” (VEGAN for short) but instead of getting a request from VEGAN to access my files, the request came from Transloadit. Call me a chicken but I don’t want any beef with users trying to use my VEGAN app.

Default oauth screen using Transloadit’s Google API credentials

desired oauth screen

[P-Riddy]

I’m also looking for tips on how to tackle Google’s Tier 2 Security Assessment. I’d love to hear from anyone that’s had success with this. Thanks!

[cognettings]

Hey, I’m still trying to get this sorted out. Have you had any luck? Where abouts are you in regards to integrating Uppy?

[cognettings]

Hey all, it’s been a while! I decided to go a slightly different route. Instead of using Uppy’s implementation of a Google file picker, I’m using the Google File Picker API.

The Google Picker API uses a more limited scope and does not require the security assessment.

So basically,

1. user clicks on the Google Drive icon in the Uppy dashboard which instead opens the Google Picker UI
2. user signs in to their Google account (access token is noted)
3. user selects files from Google Picker and clicks submit
4. the data gets massaged a little so it can be given to Uppy and shown in the Dashboard
5. user clicks the Uppy upload button
6. a custom Uppy plugin is used to upload links to the Google Drive files and the access token to my PHP server (instead of the Companion server)
7. my server then downloads the files, processes them, then uploads to S3

Uppy and Companion got me most of the way to what I needed, but I needed to write a custom solution for Google Drive. It performs well enough for now. Let me know if anyone would like more details on the implementation.

[thmclellan]

Wow, nice work @cognettings on implementing the file picker API, thanks for sharing!

I’ve been using Uppy with self-hosted Companion Drive for 4 years and just got the notice from Google that we need to move to filepicker or go through their security verification process. It seems like we have 90 days to comply.

Based on the steps you’ve outlined, it sounds like the File Picker returns some file meta data, file link, and an access token. With the link and access token, are you doing the actual file transfer on your PHP server so the file is sent from Drive to S3 via server, rather than retrieving the file to client-side and uploading back to server?

We’re using Uppy as part of a mobile app and also a web experience, so it would be nice to keep things server-side if possible. Great to know that the file picker approach is possible.

Are there any other tips you’d recommend when implementing? We’re on an older version of Uppy with a node express server and using S3. I’m guessing we could do a similar thing to what you did in PHP. Thanks

[cognettings]

Thanks I’m glad to finally have something that’s working

Yup, the transfer is taking place server side. The token and Drive meta data get posted to my server which does the downloading and transferring. However, to get thumbnails to show in the Uppy Dashboard I did do a client side request to get them (“document” being what the Picker API provides):

// get thumbnail links for all selected files
const requests = documents.map(document => {
  return fetch(`https://www.googleapis.com/drive/v3/files/${document.id}?fields=thumbnailLink,id`, {
    headers: {
      Authorization: `Bearer ${_accessToken}`
    }
  })
  .then(response => response.json())
})
const responses = await Promise.all(requests)

Then, converting the documents to Uppy files:

documents.forEach(document => {
  uppy.addFile({
      name: document.name,
      type: document.mimeType,
      data: {}, // file blob
      meta: {
        // optional, store the directory path of a file so Uppy can tell identical files in different directories apart.
        relativePath: document.id
      },
      source: 'GooglePicker', // optional, determines the source of the file, for example, Instagram.
      isRemote: true, // when true, will use the preview field to show a thumbnail in dashboard
      preview: document.thumbnailLink, // from `responses`, but added to the document
      remote: document
    });
  })

I ended up writing a custom uploader to handle the documents from Picker. The uploader gets swapped out using Uppy’s onBeforeUpload hook (Though I guess the plugin could just ignore files it doesn’t care about).

onBeforeUpload(files) { // if necessary, change which uploader to use
    console.log('onBeforeUpload')

    // check which uploader needs to be used
    files = Object.values(files)
    const file = files[0]
    let requiredUploaderType
    if (file.source === 'GooglePicker') {
      requiredUploaderType = 'GooglePickerUploader'
    } else if (file.isRemote) {
      requiredUploaderType = 'RemoteUploader'
    } else if (!file.isRemote) {
      requiredUploaderType = 'LocalUploader'
    }

    // remove previous uploader
    const uploader = uppy.getPlugin('uploader')
    if (uploader && uploader.type !== requiredUploaderType) {
      uppy.removePlugin(uploader)
    } else if (uploader && uploader.type === requiredUploaderType) {
      // update folder in case user opened a different one since previous upload
      uploader.setOptions({
        folderId: getActiveFolderID()
      })
      return true
    }

    // select uploader based on the selected files
    if (requiredUploaderType === 'GooglePickerUploader') {
      console.log('use googlepicker uploader instead')
      uppy.use(GooglePickerUploader, {
        id: 'uploader',
        authToken: accessToken(),
        folderId: getActiveFolderID()
      })
    } else if (requiredUploaderType === 'RemoteUploader') {
      console.log('use remote uploader')
      uppy.use(RemoteUploader, {
        id: 'uploader',
        accountId,
        companionUrl,
        folderId: getActiveFolderID()
      })
    } else if (requiredUploaderType === 'LocalUploader') {
      console.log('use local uploader')
      uppy.use(LocalUploader, {
        id: 'uploader',
        folderId: getActiveFolderID()
      })
    }

    return true
  }

And here’s the plugin for reference:

import Uppy from '../../../../../serve/vendor/uppy/dist/uppy.js'

/**
 * An Uppy plugin to upload files from Google Picker. Files are selected using
 * the Google Picker API to then be uploaded to our CDN (via the asset controller).
 *
 * This class is only responsible for taking the files picked and passing them to
 * the asset controller.
 */
export default class GooglePickerUploader extends Uppy.BasePlugin {
  constructor (uppy, opts) {
    super(uppy, opts)
    this.id = opts.id || 'GooglePickerUploader'
    this.type = 'GooglePickerUploader'
    this.authToken = opts.authToken
    this.folderId = opts.folderId
  }

  install () {
    this.uppy.addPreProcessor(this.prepareUpload)
    this.uppy.addUploader(this.upload)
    console.log('PickerUploader install')
  }

  uninstall () {
    this.uppy.removePreProcessor(this.prepareUpload)
    this.uppy.removeUploader(this.upload)
    console.log('PickerUploader uninstall')
  }

  // recommended way to define functions for hooks: https://uppy.io/docs/guides/building-plugins/#upload-hooks
  prepareUpload = (fileIDs) => {
    console.log(this) // `this` refers to the `MyPlugin` instance.
    return Promise.resolve()
  }

  upload = (ids) => {
    console.log('google picker upload', ids.length)
    const files = ids.map(id => this.uppy.getFile(id))

    const uploads = files.map(file => this.uploadDocument(file))
    return Promise.all(uploads)
  }

  uploadDocument (uppyFile) {
    this.uppy.setFileState(uppyFile.id, {
      progress: {
        uploadStarted: Date.now(),
      }
    })

    return new Promise((resolve, reject) => {
      const form = new FormData()
      form.set('oauthToken', this.authToken)
      form.set('documents', JSON.stringify([uppyFile.remote]))
      form.set('folderId', Number.parseInt(this.folderId))
      fetch('/asset/uploadGoogleDriveFile', { method: 'POST', body: form })
        .then(response => response.json())
        .then(responseText => {
          console.log(responseText)
          if (responseText.error) {
            this.uppy.setFileState(uppyFile.id, {
              error: responseText.error.message
            })
            this.uppy.info(`${uppyFile.name} failed to upload: ${responseText.error.message}`, 'error')
            reject()
          } else {
            this.uppy.setFileState(uppyFile.id, {
              progress: {
                bytesUploaded: uppyFile.size,
                percentage: 100,
                uploadComplete: true,
              }
            })

            // update total/overall upload progress
            const filesCompleted = this.uppy.getFiles().filter(file => file.progress.uploadComplete).length
            const numFiles = this.uppy.getFiles().length
            this.uppy.setState({
              totalProgress: Math.round(filesCompleted / numFiles * 100)
            })

            resolve()
          }
        })
        .catch(error => {
          console.error(error)
        })
    })
  }
}

Hopefully this helps. Best of luck!

[thmclellan]

@cognettings Thank you so much!! That’s fantastic, really appreciate the walk through and code, it will make it a lot easier to implement. Uppy has been a great solution so far… this custom plugin approach to supporting the file picker workflow is an awesome workaround.

[thmclellan]

Hi @cognettings I got it partially working so the user can authenticate with Google, choose file(s) through Drive Picker, and send the selected fileId to our server so we can download server-side.

For some reason I’m getting a 404 Not Found when trying to download the file using the Google Drive V3 REST API (files.get) on the server side. We’re using the same accessToken for Drive Picker and server-side downloading (with scope of https://www.googleapis.com/auth/drive.file). For now I’m just trying to get this working with regular files (apparently Google Docs / Sheets etc require the export API call).

Can I ask what scopes you’re using when trying to download the file on the server-side and what type of OAuth flow?

I’ve tried just using the accessToken but it seems like it’s not enough to download actual file content through their REST API. I’ve also tried calling the REST API directly from browser-side but that didn’t work either. I wondered if the accessToken might be limited to a single use or something.

I think my next step will be to try a different OAuth Flow (currently testing with implicit grant where we get an access token). I’m going to try using a refresh token to generate a fresh access token before trying to make the download call on the server. Anyway if you have any other tips then please let me know, appreciate any suggestions. Thanks

[cognettings]

Hmm, I don’t recall seeing any 404s when I was setting this up, I’ll have to dig through my notes in a bit.

I used the https://www.googleapis.com/auth/drive.file scope on the front end:

const tokenClient = google.accounts.oauth2.initTokenClient({
  client_id: CLIENT_ID,
  scope: 'https://www.googleapis.com/auth/drive.file',
  callback: accessTokenReceived
});

I’m using the token flow(sounds like it’s the same as you’re doing). The call I make to show a consent screen where the user can select an account is:

tokenClient.requestAccessToken({prompt: 'consent'});

On the server I make a request to the URL: https://content.googleapis.com/drive/v3/files/[DOC_ID]?alt=media&fields=* with the HTTP header “Authorization: Bearer [ACCESS_TOKEN]”. Note the “alt=media” query param is needed to actually get the file contents.

Also worth noting that I’m able to use the access token for multiple requests and didn’t need to refresh it on the server.

What’s the call you’re making to download the file (server side and client side)?

[thmclellan]

Thanks @cognettings, that is really strange. I was calling “GET https://www.googleapis.com/drive/v3/files/[DOC_ID]?alt=media” with header of “Authorization: Bearer [ACCESS_TOKEN]” on both server and client side.

I changed it to GET of “https://content.googleapis.com/drive/v3/files/[DOC_ID]?alt=media&fields=*” but still getting 404.

For DOC_ID, I’m using the “id” field of each doc in the “docs” array returned from the picker callback. The docs are just normal files on Google Drive, not shared or anything.

I’m wondering if it’s something to do with our OAuth flow. We’re using a server-redirect approach to get the access token because we support web, iOS app, and Android app users. I’m going to try changing to the standard token flow setup like you’re doing and see if that makes a difference, will report back here.

[thmclellan]

I found the underlying issue. To make it work with just the drive.file scope, I had to call setAppId(“123456789012”) while creating the picker, where the app id matches the first portion of the client id (e.g. ‘123456789012’ in ‘123456789012-sh3k8din9f1pr28q7cofxqcv6d4s84tn.apps.google etc’)

With the modified picker code, I’m able to retrieve the selected files correctly through the Drive REST API.

const picker = new google.picker.PickerBuilder()
        .setAppId("123456789012")
        .addView(targetView)
        .enableFeature(google.picker.Feature.MULTISELECT_ENABLED)
        .setMaxItems(10)
        .setSize(targetWidth, targetHeight)
        .hideTitleBar()
        .setOAuthToken(accessToken)
        .setLocale(this.context.currentLang)
        .setDeveloperKey(publicGoogleDrivePickerKey)
        .setCallback((data) => {});

Thanks to a 10 year old SO post for explaining: javascript - How do I use Google Picker to access files using the "drive.file" scope? - Stack Overflow. The “setAppId()” function isn’t part of the Builder tutorial but it’s in the picker SDK docs.

This is working with both the Google oauth client and the server-side oauth flow.

By the way, while testing with the Google oauth client, requesting just the drive.file scope, it turned out the initTokenClient() method returned an accesstoken that included other pre-existing scopes from my account including auth/drive. It made me think I had it working for a minute before I figured out setAppId.

For anyone testing this stuff, I’d recommend resetting your permissions at http://myaccount.google.com/connections and confirm it still works.

This stuff is tricky. Thanks again @cognettings for the help in troubleshooting.

[cognettings]

Awesome, I’m glad you figured it out!

I didn’t even think to check the PickerBuilder since you were able to view and select files with it. Wow, that’s so strange…

Haha, yeah definitely tricky and patience-testing. Hopefully our efforts will help someone else in another decade.

[thmclellan]

Haha yeah hopefully this thread helps other Uppy users getting that 90 day warning email from Google. Cheers!

[danielbevan]

I created an account on this forum just to thank the two of you @thmclellan @cognettings

I had the Google Picker working but with only the unrestricted scopes, we (the app) were only able to work with files that we (the app) had created for users.

We spent weeks going back and forth with Google to get the restricted scope we thought we needed.
https://www.googleapis.com/auth/drive

We were then told we needed this Tier 2 Security Assessment. At that point I thought there had to be a better way. That’s when I found this thread.

Including .setAppId() into the Google Picker configuration solved that problem entirely. Doubled checked my work just to be 100% sure.

Thanks again!

[thmclellan]

Thanks @danielbevan, glad it was useful for you!

Not sure if you have many mobile users, but after moving to the Google Picker, we noticed a couple issues where the picker didn’t well for mobile when choosing folders or multiple files. I logged both issues with Google (and if you have them too, please give them a +1 so they might get more Google attention):

https://issuetracker.google.com/issues/334994030
https://issuetracker.google.com/issues/338250108

Good luck with your project!

[cognettings]

Thanks! Haha, I’m glad our struggles were able to help out a bit

I haven’t run into any issues with the solution yet, but I’m not using it on mobile devices.

Best of luck with your project!