How to power up Quick Base Workflows and Integration with Pipelines

image for How to power up Quick Base Workflows and Integration with Pipelines

Introducing Quick Base Pipelines?

Pipelines, a relatively new addition to Quick Base, provide advanced capabilities for workflow, process automation, and integration, giving companies functionality on-par with leading integration platform-as-a-service (iPaaS) solutions. Quick Base account holders get capabilities similar to Workato, Zapier, or Microsoft Flow without requiring additional technologies or licenses.

 

Pipelines offer a perfect complement to Quick Base because, in the same way that Quick Base allows fairly non-technical users to rapidly take a business concept, process, or requirement and turn it into a functional software solution with little or no code, Pipelines simply require configuration instead of development. Pipelines work with the same rapid development paradigm Quick Base with a focus on workflow automation and integration.

 

What can Quick Base users do with Pipelines?

The principal use case of Pipelines centers around extracting data from a system outside of Quick Base and loading it to a Quick Base application or vice versa without requiring complex development.  However, neither the source nor destination systems need to be Quick Base, so Pipelines can serve as a standalone iPaaS solution to integrate and automate interactions between pretty much any different systems that can be connected.

 

Pipelines also provide powerful capabilities for handling ETL (extract transform and load) operations, where data from one system needs to be substantially modified before loading to another.  For example, if extracting some data like time-cards from QuickBooks Online to load to QuickBase requires manipulation of the data to properly match the target data or enhance the business process, Pipelines enables features to easily handle the data transformation. So not only am I pulling time cards from QuickBooks online and pulling them into QuickBase.  Transformations can be anything from switching people from last name, comma first name to first name, last name with no comma, or converting currency, or more complex operations.

 

Either to support process automation, integration or  just to do things like data cleanup, or manipulation and provide analytics on data, Pipelines can support a wide variety of needs that substantially enhance the power of Quick Base.

 

Example use cases for Quick Base Pipelines

Outlook Calendar Integration

A common use case is to leverage the Outlook Channel to connect to a specific Calendar.  For example, someone may be using Quick Base as a CRM, wants to schedule a follow-up meeting with a prospect and capture the activity without double entering data.  Pipelines allows the record to be entered in Quick Base, associated with a date, time, and a person, and then automatically generates the calendar event and invitation in Outlook.

image for Outlook Calendar Integration

 

 

image for Outlook Calendar Integration 2

Outlook Email Integration

Another great use of the Pipelines Outlook Channel uses the Email connection to ingest messages into Quick Base.  Perhaps a business is using Quick Base as an IT or technical support team to manage help tickets and want to support Ticket submission via email.  They can set up a Support email and easily automate Ticket creation from emails sent in.  An extension of this might be to use the Slack Channel and automatically post a message to the Support Team alerting them when any new help request comes into queue.

 

Slack Channel Integration

Pipelines come with a Slack Channel out of the box, which can be very useful to compliment many processes.  Slack has become very common with Corporate and Small Business teams, providing a wide range of benefits, though it can sometimes create a sense of operating outside of the business information systems.  To make Slack more complimentary to structured tools, integrating data, process, and messaging can tie everything together.  For example, a step can be added into a workflow to notify certain people or teams via Slack and provide some essential information in the message.  It’s also possible to drive a Slack Bot to take certain specific actions and even capture some of the dialogue back into Quick Base.  In this way, Slack can be more effective and operate as a centralized service for ad hoc chatting as well as alerts about relevant business process updates.

 

Quick Base Pipelines Basics

 

Getting Started

The Pipelines feature of Quick Base operates like an independent application (it’s not a feature inside one of your existing applications) and can be found under My Apps where users will see Pipelines as an application you can select to access.

 

When users are ready to start using Pipelines, they can just access it from the Menu and dive in.

 

Pipeline Configuration

Before starting to build Pipelines it’s necessary to set up some channels which will be needed to connect to various data or services.  When adding the Channels users need to authenticate with the external service and then they will be available to use them in any of their Pipelines.

 

image for Pipeline Configuration

 

Once the desired Channels have been enabled, from the Pipelines application users can simply create a Pipeline to begin configuration.  A key initial decision will be to decide whether the Pipeline will be triggered or scheduled.  Triggered Pipelines will fire when a certain event occurs in QuickBase or one of your Channels, such as an email arriving or a change in some data.  A scheduled PipeIine will fire at a specified interval of time, such as hourly, daily, etc.

 

In the case of a triggered event in an Outlook Channel, users are able to select the specific trigger from a list of possible triggers for the channel and then complete any specifics required to detail the event.

 

After configuring the trigger, users then need to specify what will happen with the data received from the source system, in terms of how the data might be manipulated (more details on this later) and where it will be sent by adding steps in the Pipeline.

 

image for pipeline

 

Deploying Pipelines

It’s important to understand that Pipelines are currently tied to a specific Quick Base user as the Pipeline owner so there isn’t a current model for managing group access or control for deployment.  Administrators are able to see an overview of Pipelines running and perform some basic control features.

 

On the flip side, testing of Pipelines provides robust features to step through each stage of the Pipeline and see the input and output of the stage.  This makes it easy to see what data is being received from a source system, how that data appears after any transformation, and what data is being sent to any target systems.  For example, when loading in an email users can see the raw subject, body, to, from, CC, etc.  Once that data is pushed to Quick Base the API response can be viewed to verify the record was properly added or address any errors, right down to the field to field mapping.

 

image for Deploying Pipelines

 

Managing & Monitoring Pipelines

The Pipelines application provides a dashboard upon accessing which shows all of the currently operating Pipelines with real-time updates when triggers fire and an activity feed showing recent actions.  For example, if a QuickBooks job just initiated users will see an alert like: “QuickBooks invoice sync just started,” and users can essentially watch the process run in real-time.

 

image for Managing & Monitoring Pipelines

 

In addition, each Pipeline has its own Dashboard with an activity log that users can drill into, see the history of execution and even drill further down into the individual inputs and outputs of every single step.  This level of logging and monitoring on each Pipeline provides a powerful feature for checking on the stability of the processes and troubleshooting if issues occur.

 

processes and troubleshooting if issues occur.

 

The main Pipelines dashboard and each individual Pipeline dashboard also provide details on the metering related to account billing as Pipelines are charged in Quick Base on a consumption basis.

 

Pipelines are charged in Quick Base on a consumption basis.

 

Insider Tips

 

Using Jinja

Many Pipelines can be built without requiring much technical experience.  However, more complex ETL transformations and logical operations may require some more advanced skill.  Pipelines use a Python-based syntax for manipulating data called Jinja.  Jinja supports simple yet powerful capabilities to operate on source data and transform using calculations, logical statements, and even summaries.

 

For example, a fairly non-technical user could quickly grab emails and send them to Quick Base or Slack. However, the email content might have some unwanted content such as HTML tags, etc. which might require some manipulation with Jinja to strip out the junk.

 

Jinja operates similarly to SQL and can offer very powerful means to search datasets, query, summarize, aggregate, and more.

 

Getting familiar with Jinja can unlock some pretty amazing opportunities with Pipelines.

 

https://jinja.palletsprojects.com/en/2.11.x/

 

Using REST API’s

While pre-built Channels will help users connect to many popular applications, numerous others will need to be accessed via the Webhook Channel which requires some understanding of how to authenticate and operate with Rest APIs.

 

Most SaaS and internet applications provide Rest APIs as a means to integrate with the data.  However, the methods for authenticating, querying, and posting data can vary substantially.  Studying the source system API’s and getting familiar with the common approaches to Rest API’s will likely be valuable time spent for users developing a variety of Pipelines.

 

https://tray.io/blog/what-is-an-api-integration-for-non-technical-people

 

Using a Quick Base Partner for Help

With the low/no-code promise and power of Quick Base, many users will develop powerful applications with little or no professional assistance.  As applications become more mission-critical and requirements more complex, professional services from a Quick Base partner may be the difference in achieving long term success.

 

A certified Quick Base partner such as MCFTech can help implement advanced Pipelines with complex ETL transformations, help integrate with systems where out-of-the-box Channels aren’t available by leveraging extensive API experience, and design automations based on best practices from many years helping clients succeed with Quick Base.

 

In addition, MCFTech can assist with new advances as Pipelines evolve such as the upcoming release of an ODBC channel and onsite client, which will securely allow Pipeline integration with on-premise Corporate data in SQL and other legacy data systems.

 

Contact us more to learn how we can help automate your business.

Contact Us