So, the last few posts were my attempts to try to make sense of the random ideas that came up from several different conversations over the past week. I probably should have prefaced those posts with a bit of a warning since I know it confused some people. I think I’m now seeing the wisdom of having a personal wiki for collecting these thoughts, organizing them and inviting collaboration. Perhaps that can be part of my personal knowledge management wiki, but I’m not sure I’m comfortable with private data in the same system that’s on the internet, but that’s a completely different project.
Regardless, lacking a good mindmap solution… That was my attempt at explaining a complicated problem and trying to explain all the little parts that would make up a solution… or at least how I would envision a solution with today’s technology…
First of all, this project isn’t about creating a business or otherwise sell this idea. There’s nothing stopping someone from providing a service with this software, but I at least want the option of being able to run this myself on my own machine, as I’m leary of handing over access to all my personal data.
With that said, I’m envisioning a HUGE system with tons of functionality. I’d like to build it all myself, but I know that I don’t have time for that and some of the items are outside my areas of expertise. I want anyone to be able to run this software on their own computers on pretty much any operating system and any type of internet connection. So, I’m seeing this as a GPL’ed open source python project and hopefully, some people will catch wind of this and want to help out! I still have to sketch out a roadmap of features and functionality, but the core design is already pretty well-formed.
Central to this system is the main engine that will power the virtual digital assistant… the brain that allows it to fetch/receive input, process it according to the rules its been given (including any self-learning AI), and then act accordingly. As complicated as that system is, there’s only a handful of functions that the “brain” will have to do. Using a modular design allows for flexible use and extension of the system, allow this project to become a development platform allowing it to act like yahoo pipes.
Minimally, i can see the following modules being needed, in addition to a little bit of python glue to connect everything together, and there would be a working solution.
Inputs
Without input, our virtual assistant is deaf and blind. There’s really two ways for input to be received… either we need to go out and get it, or its going to find its way to us. Examples of very “pull” inputs are RSS feeds or email notifications (via POP/IMAP polling), while “push” inputs may be generated by things like Instant Messenger Bots or a web interface or a phone app. These input modules are responsible for doing what it needs to acquire or listen for data, processing it and creating core events.
Key to this system to be successful will be the definition of these core events. They should be simple and general enough so the basics are handled well. At the same time, they are extensible so that new events can be created. For example, a basic event may be a “blog entry object” generated from a RSS Input Module. Perhaps a Twitter input module will read your twitter stream and create Tweet objects. These likely are all subclassed from an Event class that provides the basic functionality needed for basic processing and it would be recommended to avoid excessive subclassing.
Processing
Once we have the inputs and have generated events, all the real work has to happen. Ultimately, this is where all the exciting things happens. These processing modules are chained together. Perhaps one module will look at the the content of an object and classify it by looking for keywords, while another module will use that classification to look at your current Attention Profile and decide if an alert should be generated or not. Initially, these can be very limited in functionality while still providing a very usable system. At a minimum, the Processing modules will need to generate Output Actions. An example Processing module may read a Tweet Event and generate a notification output object if it meets some requirements. Occasionally, I may want to temporarily stop following someone on twitter because they are tweeting a lot more then normal about things I don’t care about… so for the next six hours, I’ll only get notifiied if they @reply to me.
Outputs
Finally, Output actions actually take these events and do something. Perhaps it sends an email or text message… or uses the Remeber the Milk API to add or mark a task as complete… or maybe it sends a DM through twitter on my behalf… or adds something to my Google Calendar. Today, most all of these actions are easily automated. Also, its important to note that output notifications should provide a feedback loop to generate additional input events, such as marking an item as read in my google reader via an API call and also adjust my attention profile accordingly.
So that’s an overview of the system. Given its flexibility, configuration and management of installed components may be difficult, but an average desktop machine is more then powerful enough to automatically do these tasks in the background.
With this system, your personal assistant can sit in an IRC channel and be told to notify you over the weekend when some keywords are said in a channel, or be told that you’re expecting an email from someone in the next 3 days and if you don’t get one you want to be reminded with an email to your work address. The later example will set a pending event to notify you of the missing email, while scanning your mail box via IMAP and if it finds an email it clears that pending notification. Or perhaps, the agent can be a simple security system by accessing a webcam image internal to my network and watch for motion and capture video/images and save them for my review later.
I haven’t yet found a platform that can even begin to handle that wide array of tasks in stride, let alone be expanded to handle the really slick things like predicting what I’d be interested in reading or attempting to keep my life on track by ensuring that my goals aren’t being neglected.