Composable Analytics

[MUSIC PLAYING]

ANDY VIDAN: I'm Andy Vidan, and I'm the CEO and co-founder of Composable Analytics. I've spent the last 10 years, almost, at MIT Lincoln Laboratory as a technical staff member and associate technology officer. My background is in physics and math, and I at the laboratory, I studied distributed computing and built web applications for Homeland Security and defense systems.

LARS FIEDLER: So my name is Lars. I'm the CTO and co-founder of Composable. Andy and I founded this about a year ago, where we were at MIT Lincoln Laboratory working together on this project. At the lab, I was working on various distributed command and control systems, [INAUDIBLE] gaming platforms. I have a pretty extensive background in business intelligence. So I was a software developer at Microsoft.

ANDY VIDAN: So Lars and I met at MIT Lincoln laboratory. At the lab we've, both been working on distributed computing systems, big data, Data Fusion projects for both Homeland Security and defense applications. And my background is in math and physics, and Lars' background is in software development.

LARS FIEDLER: Yeah, that's right. So when we were at the lab, we recognized that there was a lot of problems in the industry in terms of data and managing that information, and in linking people together across those different information sources that they use day to day. And so we decided to try and tackle that problem and provide a general and flexible solution to manage the flow of information between people and these complex data systems.

ANDY VIDAN: So MIT Lincoln Laboratory's at the forefront of providing big data solutions, data analytics to our defense and Homeland Security personnel. And we saw early on that there is lots and lots of data coming to an army of analysts, intelligence analysts, security analysts, defense analysts, who needed a self-service platform to be able to conduct their own data analysis on the data that they're receiving. And because of that, we developed a very flexible solution, as Lars was saying, that allows an analyst to, within a single platform, do the entire analytical pipeline of data ingestion, parsing, enrichment, cleansing, analysis, and finally distribution. And this tool was innovative and one-of-a-kind, and because of that, we saw that there is applicability within the private sector for this tool.

LARS FIEDLER: Yeah, and so one of the initial use cases for the platform was really in the health care arena. And primarily in public health, although it's being applied in many other sectors now in terms of the energy sector and finance as well. But in terms of the health care sector, one of the things that we see is that day in day out, people are trying to do and solve the same problems of getting data, analyzing data, pivoting in various ways, and basically, going through and mining all this information. And so we decided, let's create a platform that allows them to do that very seamlessly all the way, as Andy was saying, from ingesting the data, massaging it, all way up to visualizing that information in an intuitive way.

LARS FIEDLER: When you're designing a system where you usually go to a whiteboard and you start to draw boxes and link them together and figure out who's going to be talking to who, who requires certain information, where you're going to get this. And it kind of stops there.

And the development arm comes along and looks at that. And they start coding. And you've lost that connection from this nice, visual diagram to the actual code in the platform. When you actually decide to change things or figure out how the system is actually working, well, it's buried in all of this code.

And so what we've done is we've provided this data flow methodology that allows you to very easily string together components in a visual manner in a designer in order to create your workflows and adjust them as necessary. So rather than your system is being very stagnant and tightly coupled, with Composable, you're-- it really allows companies to adjust to the various markets if they want to enter a new sector. It allows them to very easily adjust their human workflows and their data workflows and bring in more information and go after new markets.

ANDY VIDAN: So the core of that innovation was to bring together the data flow visual programming interface with data science, making data analytical modules available within these applications, and allow the analysts and the users to be able to have a self-serve platform for data analytics. Think of a flow-based diagram for how a program should function.

LARS FIEDLER: So with data flow, the-- rather than having the data sit in one single database, what this allows you to do is move the data. So as it's coming through, you can cleanse it and push it out to the various systems. And there's a lot of benefits with that. One is, you get the modularity in the various stages that are required to manipulate the data.

But it also is much more [? pro forma ?] as well. So and the system can basically look at what are those various stages and then things that have to happen within the workflow in parallel [INAUDIBLE]. So not only are you getting just in terms of time in terms of the execution of your workflows is decreased, but also the development time of those is also decreased.

For example, one of our customers in the financial sector, basically one of the first things that were tried to tackle is lead generation for them. And at the time, they only had a couple of marketing folks trying to bring in various leads. And so when we looked at the problem with our solution, we were able to basically scale them up 10x by ingesting massive amounts of data, cleansing it, and also extracting the necessary bits out of it and prioritizing their leads. And so by using our system, we effectively grew their business 10x in about three months.

ANDY VIDAN: So at a higher level, if you think of an organization that's very data driven, they're getting data from lots and lots of sources. And they might be coming in via emails, via FTP, via third-party databases. And typically they'll have a single person or single analyst there to grab that data out of that data source, transform the data, cleanse it in some way, and push it into their own data warehouse.

And here that we're able to automate that process, again, using this data flow approach where specific modules go out to the FTP sources. They check the emails. They check the shared drives. They check the third-party databases and bring in that data in an automatic fashion and run it through in a streaming process, through a cleansing algorithm, through other analytics, finally through the endpoint, which could be a database insert. And in this way, it liberates the staff of that organization to focus more on the analytics rather than on the process of bringing in the data.

ANDY VIDAN: Composable Analytics is a platform. It's meant to be a self-service platform so a business user can run advanced analytical applications. It's a very modern, intuitive designer to create your own flows of data.

We also allow through these unique modules some advanced algorithms so that people can mine the data. They can fuse the data. They can match data one data set to another data set and be able to really gather and extract some insight out of the data sources.

LARS FIEDLER: You can basically go to our website and download our installer and get it on your laptop in a couple of minutes. The entry into using Composable is very, very low. We also have a cloud-based version out there where you can just log in and start using it.

It is a very flexible system and can be customized to suit your needs. So we ship with the very general modules and capabilities. But you can always plug in your custom code for your business needs and your workflows.

ANDY VIDAN: Out of the box, there's an entire library of modules that do anything from ingesting data from databases from Excel spreadsheets, from HTML sources, to doing OCR, to clustering algorithms, regression, and finally all sorts of visualization, whether it's geospatial visualization or basic scatter plots.

So having technical backgrounds, we've always been impressed by the power that software development and data science gives us. So we can analyze all sorts of data. We can write programs to do all sorts of things.

And the impetus behind providing Composable Analytics as a platform to business users is to really empower them to not require any custom software development that takes a long time, but rather providing them with a visual platform to empower them to do their own analytics.

And moreover, collaborative analytics is a very big topic for us. And it's a very big focus area. And here where one analyst can design his own analytical application, his own data flow application, he can now share that with other people in his organization who can clone it, can modify it, can adjust it to their needs, and be able to build on top of their colleagues' applications.

LARS FIEDLER: Really the platform is all about sharing your work and sharing your analytical data flows. So if we're to compare this to, say, maybe something like Excel where all of your algorithms and your pivots are very tightly coupled to the data, well, with Composable, I can develop an algorithm. And then I can develop data flow that's separated from the data and share that out. And someone can feed additional information through it and reuse that capability very easily.

ANDY VIDAN: The core building blocks of our applications are these modules. Modules have inputs and outputs. Inputs are data sources that come in to the module. And outputs are some output that was acted upon within the module, whether it was an analytical module, or a cleansing module, or some type of data enrichment module.

Because these-- the steps within your application is broken up into these core modules, each module can run on its own as soon as its inputs are satisfied. So in this way, the data flow algorithm that you author as a user is inherently parallel.

LARS FIEDLER: You need to decide where it's actually going to run. So the system basically abstracts out where your modules and where your code is physically going to run. And so by doing that, we can basically execute these applications across hundreds of different nodes out there in a cluster.

ANDY VIDAN: So as an example of what the Composable Analytics platform can do, it can empower a business user to run very complicated OCR algorithms. So OCR is Optical Character Recognition. It's when you try to extract digital text or characters out of an image file.

So take a scanned PDF document. It might have some very nicely typed text in there. It might also have handwritten notes or check boxes that are checked with a pen. So we can create-- a business user can create an analytical flow that takes that PDF document and runs it through an analytical application where part of that analytical application would extract out the text in there.

Another part might extract out areas within the document that can then be streamed into yet another module, which would do more advanced optical character recognition on the check boxes or the handwritten notes and finally join all that together and input that into your final database or a spreadsheet.

LARS FIEDLER: And once you've designed that flow and have executed it on one, the system is very scalable. So it can scale out to hundreds and thousands and millions of documents. And also it's scaling in terms of first initially trying to extract out all of the necessary information. But when it can't, it can farm out and distribute those errors or those things where it wasn't able to extract out 100% of the information to individuals. The time it takes to actually implement this distributed system, is very, very low.

ANDY VIDAN: So the example we just spoke about is a working example for a very large insurance company here in the Boston area. And they have half a million-plus files that come from lab reports, doctors' notes that they have to transcribe into a digital form so that they can run their analysis and be able to underwrite insurance policies. So in this way, they are able to streamline the entire process.

As another business example of a use case for Composable Analytics is exploration of minerals. There are many organizations out there who are exploring areas to find minerals, whether it's copper, oil, gas, natural gas. And there are various ways of doing it. One way, which is the industry standard unfortunately, is to randomly drill in areas and be able to figure out whether there is anything there or not.

Now with Composable Analytics and with the availability of data, whether it's public data sources about what-- who drilled what in the past or data directly from geophysicists who can now place their data in geospatial views and on maps, a user within an organization can now grab all the variety-- all the varied sources of data, bring it into one system, and be able to run their analysis to be able to inform them on the best places to drill.

No transcript