Adonit Jot Touch 4 Support for Brushes iPad App

image2

And now for something completely different…

With the announcement of Adobe’s teaming up with Adonit to help manufacture and ship Mighty & Napolean in the first half of 2014, I thought it might be a good idea to start getting my head wrapped around how all this technology works.

I recently heard a great interview with British artist David Hockney about the incredible art he is creating on the iPad.  Unfortunately, I wasn’t able to make it out to his “A Bigger Exhibition” at the de Young museum in San Francisco, which just ended. One thing that did catch my attention in the interview though, was his mention of an app called Brushes that he uses to create his art:

image2

 

Those are just my scribblings.  Not to be confused with anything closely resembling art.

Upon further investigation, I discovered that the Brushes app is in fact open source and available on GitHub.  Brushes as it currently stands, supports the Pogo Connect pen.

In order to give myself a  good project to learn more about the Adonit Jot SDK, I forked the Brushes app and added support for the Adonit Jot Touch 4 stylus. It turned out to be much more straightforward than expected, once I got my head around the Brushes source code. You can find the results here on GitHub.  It’s probably not quite ready to be merged back into the original Brushes app, but it’s definitely a great start.

If you’re interested in what exactly I did, keep reading…

Installing the Jot Touch SDK

After cloning the Jot Touch SDK from GitHub, I copied the JotTouchSDK.framework folder to the Brushes app folder and then opened Brushes.xcodeproj. Then, I followed the instructions in the Jot Touch SDK Getting Started Guide to import the Jot Touch SDK into the app. Just read the guide for that.  There’s no reason for me to repeat some very good documentation.  The two main steps to cover are:

  • Link your project with the CoreMotion.framework, CoreBluetooth.framework and JotTouchSDK.framework frameworks
  • Add ‘-ObjC’ to the Other Linker Flags

I did have to make two minor tweaks to the Brushes app to get it to compile:

I was getting a “No newline at end of file” for JotStylusManager.h.  Rather than changing the code, I just changed the Build Setting ‘Missing Newline at End of File’ to ‘No':

Screen Shot 2014-01-20 at 2.56.53 AM

As well, there was a minor issue in WDAddPath, line 141:

WDLog(@"Empty path bounds with path of length %d", [path.nodes count]);

had to be changed to:

WDLog(@"Empty path bounds with path of length %lu", (unsigned long)[path.nodes count]);

to prevent a casting error.

Defining a New Type of Stylus

WDStylusManager.h defines an enum ‘WDStylusType’ which originally supported WDNoStylus and WDPogoConnectStylus.  I added WDJotTouchStylus:

// Jot Touch 4
#import <JotTouchSDK/JotStylusManager.h> 

typedef enum {
    WDNoStylus = 0,
    WDPogoConnectStylus,
    WDJotTouchStylus,
    WDMaxStylusTypes
} WDStylusType;

I also defined a new stylus manager specifically for the Jot Touch:

@property (nonatomic) JotStylusManager *jotManager;

Then, in WDStylusManager.m, I needed to synthesize the jotManager:

@synthesize jotManager;

Displaying Pen Status

The Brushes app allows you to see the current stylus connection status and battery level:

image

 

In order to provide that information, the dataForStylusType method needed support for the Jot:

    ...
    } else if (type == WDJotTouchStylus) {
        data.productName = NSLocalizedString(@"Jot Touch", @"Jot Touch");
        data.connected = (jotManager.connectionStatus == JotConnectionStatusConnected);
        data.batteryLevel = @(jotManager.batteryLevel / 100.0f);
    }

The Brushes app expects a battery level between 0.0 and 1.0.  The jotManager’s batteryLevel ranges from 0.0 to 100.0 and has to be scaled appropriately.

Supplying Brushes with Pressure Data

One of the great things that separates a stylus from simply your finger for example, is the ability of the stylus to tell the app exactly how much pressure the user is exerting on the display so the app can react appropriately. In order to supply the Brushes app with that information, I had to add Jot support in the pressureForTouch method.

I tried turning the actual touch passed into the method into a JotTouch, and then getting the pressure from that, but that didn’t seem to work for some reason, so I ended up just retrieving the current pressure directly as shown below:

- (float) pressureForTouch:(UITouch *)touch realPressue:(BOOL *)isRealPressure
{
    float   pressure = 1.0f;
    BOOL    isReal = NO;

    if ((mode == WDPogoConnectStylus) && [pogoManager oneOrMorePensAreConnected]) {
        isReal = YES;
        pressure = [pogoManager pressureForTouch:touch];
    } else if ((mode == WDJotTouchStylus) && ([jotManager connectionStatus] == JotConnectionStatusConnected)) {
        isReal = YES;
        pressure = [jotManager getPressure] / 2047.0f;
    } else if (mode != WDNoStylus) {
        isReal = YES;
        // since we're in stylus mode, but no styli are active, use 1.0 pressure
    }

    if (isRealPressure) {
        *isRealPressure = isReal;
    }

    return pressure;
}

The Brushes app expects the pressure value to range from 0.0 to 1.0.  The jotManager returns values from 0.0 to 2047.0 so they have to be scaled appropriately.

Listening for Connect/Disconnect Events

Finally, we need to listen for Jot connect/disconnect events and tell the app when these events occur so it can update the display appropriately.

When the Bluetooth state is set in setBlueToothState, if the Bluetooth is on (blueToothState == WDBlueToothLowEnergy), we initialize the jotManager and listen for connection status changes:

- (void) setBlueToothState:(WDBlueToothState)inBlueToothState
{
    if (inBlueToothState == blueToothState) {
        return;
    }

    blueToothState = inBlueToothState;

    if (blueToothState == WDBlueToothLowEnergy) {
        if (!pogoManager) {
            pogoManager = [T1PogoManager pogoManagerWithDelegate:self];
            pogoManager.enablePenInputOverNetworkIfIncompatiblePad = YES;
        }

        if (!jotManager) {
            jotManager = [JotStylusManager sharedInstance];

            [[NSNotificationCenter defaultCenter] 
                addObserver: self
                selector:@selector(jotConnectionChange:)
                name: JotStylusManagerDidChangeConnectionStatus
                object:nil];
            jotManager.rejectMode = NO;
            jotManager.enabled = YES;
        }
    }

    [[NSNotificationCenter defaultCenter] postNotificationName:WDBlueToothStateChangedNotification object:self];
}

This will call jotConnectionChange whenever there is a change to the connection status.  jotConnectionChange is then defined as follows:

- (void)jotConnectionChange:(NSNotification *) note
{
    switch([[JotStylusManager sharedInstance] connectionStatus])
    {
        case JotConnectionStatusOff:
            break;
        case JotConnectionStatusScanning:
            break;
        case JotConnectionStatusPairing:
            break;
        case JotConnectionStatusConnected:
            [self didConnectStylus:NSLocalizedString(@"Jot Touch", @"Jot Touch")];
            break;
        case JotConnectionStatusDisconnected:
            [self didDisconnectStylus:NSLocalizedString(@"Jot Touch", @"Jot Touch")];
            break;
        default:
            break;
    }
}

And that’s really all I had to do.  The only other issue I had was getting the Jot Touch 4 stylus to connect with the iPad the first time.  I kept getting JotConnectionStatusPairing notices, followed by JotConnectionStatusDisconnected notices. I finally realized that the JotConnectionStatusPairing notice was telling me that it was waiting for me to draw on the iPad to confirm that I wanted to pair my device.  Once I did that, it connected just fine, and I never needed to do it again.  The device just connects automatically when I start the app.

Next Steps

The Jot Touch SDK can send you drawing and palm rejection events.  The latter can be used to ignore events generated by your palm resting on the iPad as you draw.

In the meantime, have a play around with it and let me know what you think.

Designing and Debugging Web Layouts With CSS Regions

Screen Shot 2013-10-04 at 3.07.59 PM

The adoption of CSS Regions is definitely moving along swiftly.  They are now supported in Mobile Safari on iOS7, Opera, Chrome Canary, Chrome Stable & WebKit nightly. Browser support continues to evolve and can be tracked here.

CSS Regions allow you to define content that can “flow” through a series of defined regions on the page, so you can create magazine-type layouts and responsive websites through simple CSS.  I wrote about how you can use CSS Regions for creating responsive content in an earlier article.  I’d definitely recommend reading that before continuing here if you haven’t already done so.  It should give you a good basic understanding of how CSS Regions work.

Defining one piece of HTML markup as the source for another piece of markup is an extremely powerful, but very different way of thinking about Web page layout. You’re very likely going to want good authoring tools, and the ability to debug in-browser to figure what’s going on when your site behaves unexpectedly.

What we’ll describe here are some options for authoring and debugging Web layouts that use CSS Regions.

Tooling

As far as tools go for authoring content with CSS Regions, your best and only option at this point is Edge Reflow CC, Adobe’s Responsive Web design tool. Within Reflow, you can easily lay out linked region containers exactly where you want them to be placed, define how you want the different region containers to react (resize, move, hide, etc) to different screen resolutions and/or window sizes , and then define a block of content (text, images, videos, etc) to flow into the linked regions.

Vincent Hardy, Director of Engineering for the Adobe Web Platform team, describes here in detail how CSS Regions work, and how they are supported in Edge Reflow.  As well, in the following video, Jacob Surber, product manager for Edge Reflow, shows you how to design your Web layouts in Reflow using CSS Regions:

A Virtuous Circle

What is pretty amazing here, is that CSS Regions is a W3C spec that was first introduced by Adobe.  It has received widespread support across the browser community in a short amount of time.  CSS Regions are now even supported on iOS7 mobile Safari, and thus will likely show up in the next release of desktop Safari in the upcoming release of Mavericks.

Now that it has received enough industry adoption, Adobe is starting to incorporate the feature into its design tools.  Hopefully, designers will start using this feature, and demand more design-focused features like this. The demand will produce new standards and new browser features, and so it goes, round and round. This is a really nice virtuous circle.

Screen Shot 2013-10-10 at 11.27.34 AM

In-Browser Debugging

Once you’ve authored your regions-based content and are viewing it in the browser, you’re still going to want be able to inspect your layouts and pick apart what’s going on, especially when your site is not behaving as expected. There are several in-browser options available to help you visualize and debug your layouts and explore the different content flows, depending on which browser you are using.

WebKit Nightly

CSS Regions are enabled by default on WebKit nightly, so once you’ve downloaded the latest nightly, you’re good to go. Support was added very recently to provide some very nice visualizations to help you debug your CSS Regions.

If you’re running WebKit nightly, try out the following example I have posted on this site:

http://blattchat.com/demos/regions/basic/

Screen Shot 2013-10-07 at 1.39.04 PMNotice the different connect text regions, interspersed with an image and some highlight text. The text flows from one connected text region to another, regardless of the size of the window, or the resolution of the device you are using to view the content.

In this example we have one DIV that defines all of the source:

<div id="source">
Far far away, behind the word mountains, far from the countries Vokalia and Consonantia, there live the blind texts. Separated they live in ...
</div>

In the CSS, we take that DIV out of the normal layout and indicate that all of the source content should flow into ‘main-thread’ by using the flow-into CSS attribute:

#source {
  -webkit-flow-into: main-thread;
  flow-into: main-thread;
}

Notice how we use both the -webkit- prefixed and unprefixed version of the CSS attribute.  You need the -webkit- prefix to get this to work for now, but eventually as regions become more widely implemented that will no longer be necessary.

We also have seven DIVs laid out in different parts of the page, all of class ‘region’.  In the CSS we define each of these DIVs as a region container using the flow-from CSS attribute.

.region {
 -webkit-flow-from: main-thread;
 flow-from: main-thread;
}

All of the source content in ‘main-thread’ will flow into the region containers. In this example, the source content is just text, but there’s no reason it couldn’t also include images, videos, iFrames, etc.

Now, normally if you want to see what’s going on in a page, you can use the Web Inspector. If you right-click on some of the text and select “Inspect Element”, the Web Inspector will pop up and show you that element:

Screen Shot 2013-10-04 at 2.58.09 PMNotice that it has taken you to the source text.

But with CSS Regions, the source content is kept separate from the actual layout.  The regions define the layout and the source content flows into the regions.  So how do we visualize the individual regions that make up a given flow?  What we really need is to be able to see how the content is going to flow from one region to the next.

If you move your mouse over each of the region containers (DIV elements of class ‘region’) in the Web Inspector, you will see the following:

Screen Shot 2013-10-04 at 3.07.59 PM

 

All of the connected regions in the flow are numbered (indicating the order of the regions in terms of flow) and highlighted a light blue.  Lines are drawn between each of the connected regions to show you where the next connect region is in the flow.  And finally, the currently selected region is highlighted a darker blue.

That’s definitely more useful!  In fact, if you’ve ever used InDesign, or as you saw in the Reflow video above, you’ll find that this interface will be very familiar. Now you can play with the HTML/CSS of each region or resize the page, and see how the content flowing into those region containers adapts.

Chrome

There’s a great detailed article here by Christian Cantrell of the Adobe Web Platform team that discusses debugging CSS Regions in Chrome.  Remember though, CSS Regions are disabled by default on Chrome.  You need to navigate to chrome://flags and enable ‘Enable experimental WebKit features’.  As well, to get the CSS Regions debugging support in Web Inspector, you’ll also want to enable ‘Enable Developer Tools experiments’.

Let’s now try out the same example in Chrome:

http://blattchat.com/demos/regions/basic/

You can bring up the Web inspector by right-clicking on the page and selecting ‘Inspect Element’, or pressing cmd/ctrl-I.

You can right-click anywhere in the Elements tab in the Web Inspector and you’ll see an entry for ‘CSS Named Flows…’.

Screen Shot 2013-10-04 at 3.21.56 PM

Selecting that will provide you with a list of all of the available flows and region chains:

Screen Shot 2013-10-04 at 3.23.59 PMAs you move your mouse over each of the region containers in the Web Inspector, they’ll get highlighted in the browser — very useful for debugging CSS Regions.

The tools and technology are definitely evolving.  Give it a try and let me know what you think.

Gradient Maps, now on iOS7!

Screen Shot 2013-10-04 at 4.30.12 PM

Gradient maps, they’re not just for desktop anymore…

gradientmaps.js is a library I wrote that leverages SVG Filters to enable you to easily apply a Gradient Map to any DOM element.

Screen Shot 2013-10-05 at 10.30.58 AM

Screen Shot 2013-10-04 at 4.30.12 PM

When I first announced the library, it seemed obvious to me how you could use the library for creative effects, but then I got thinking of some practical use cases for the library. Then I went through some Photoshop-type layering techniques in another post.

Up until now though, gradientmaps.js was only supported on Chrome & Firefox.  With the release of iOS7 however, the library is now supported on mobile Safari, so you can view gradient maps on your iDevices!

Here’s a snapshot from my iPad of this Codepen: http://cdpn.io/Kbkup

image

If you’re running Firefox, Chrome, WebKit nightly, Opera or iOS7 mobile Safari, you can try out the pen right here:

Check out this Pen!

This particular example has two overlapping DIVs with the same content.  The top DIV has a gradient map applied to it.  The bottom DIV is left as is.  You can then control the transparency of the top layer to define how much or how little of the gradient map you want to show.

You can find out more about the library in this post on the Adobe Web Platform team blog.

gradientmaps.js is now supported on so many browsers, there’s no reason not to check it out. The Web like you’ve never seen it before.

Clip Paths & Blending Demos Updated for Mobile, Supported on iOS7

The recent release of iOS7 brings many new exciting features to mobile Safari.  I’m going to be writing about this in more details shortly. In the meantime, I’ve updated my CSS Clip Paths and Canvas Blend mode demos to support touch events so you can try them out for yourself on your iPhone/iPad running iOS7.

Here is the updated interactive clip paths demo:

Check out this Pen!

Try grabbing the clip path points and moving them around with your finger (or of course your mouse if you’re viewing this on your desktop version of Chrome (and ultimately Safari on Mavericks).

And here is the updated canvas blend mode demo:

Check out this Pen!

Trying drawing on the image with your finger or mouse.  It also supports multi-touch.  You should be able to draw with up to 3 fingers simultaneously.

That’s it for now.  Have fun with that.  I’ll be following up shortly with a roundup of many of the new Web standards that have been implemented with the new release of Safari.

Photoshop Blending in a Canvas

Screen Shot 2013-09-16 at 11.48.56 AM

I explained a bit about compositing and blending in a previous post, how Canvas blending lets you achieve in a Canvas the same kinds of blending effects you’d find in Photoshop.

Updated – September 24, 2013:
Canvas Blending is also supported on Firefox and iOS7 Safari

Programmatically working with the blend mode is pretty straightforward.  You just take your Canvas context and set the variable globalCompositeOperation:

var ctx = canvas.getContext('2d');
ctx.globalCompositeOperation = 'multiply';

If you’re viewing this post with Chrome and have turned on experimental Webkit features, or you’re running the latest stable version of Firefox or iOS7 Safari, you’ll be able to actually play with this feature below.  Try experimenting with the blend mode and/or color and then drawing on the image.

Check out this Pen!

If the blending is not working for you above, check out this video:

 

 

Interactive CSS Clip Paths

Screen Shot 2013-09-10 at 6.05.08 PM

Did you know that on Chrome you can specify complex clip paths for your content very easily using the clip-path CSS property?

With the standard clip property, you can specify a rectangular clipping area for absolutely positioned elements, as in:

mycontent: {
   position: absolute;
   clip: rect(5px, 100px, 100px, 5px);
}

If you ask me, that’s of pretty limited use.

With the clip-path property (-webkit-clip-path on Chrome) however, you can specify any of the following 4 basic shapes for clipping:

  • rectangle(<top>, <left>, <width>, <height>, <rx>, <ry>)  – rectangle with a top, left, width, height and optional rounded corner radii
  • circle(<cx>, <cy>, <r>) – circle with a center and radius
  • ellipse(<cx>, <cy>, <rx>, <ry>) – ellipse with a center and x and y radii
  • polygon(<x1> <y1>, <x2> <y2>, …, <xn> <yn>) – polygon with a list of points

The following Pen uses the polygon option to let you interactively set the clipping on some content.  Try clicking on the image near one of the points of the clipping polygon.  The nearest clip point will track your mouse until you click again:

Check out this Pen!

You can also specify a complex clip path in some SVG content and then reference that path using:

clip-path: url(#svg-clip-path);
Check out this Pen!

Don’t forget to click on the different tabs to switch between HTML, CSS and the end result!

The creative possibilities here are endless…

It’s important to note though that:

(-webkit-)clip-path: shape

is only supported on Chrome at present, but:

clip-path: url(#svg-path)

is also supported on Firefox.

Keeping track of what’s supported where is a little tedious at best, so I’ve created the following Github repo to help keep track of the various CSS Graphics standards and in what configurations they’re supported on what browser.  Please help keeping that project up to date!

https://github.com/awgreenblatt/css-graphics

This stuff is pretty straightforward and very useful.  I can’t imagine not seeing this show up in more browsers before too long.

Adobe Create Now World Tour

creativecloud

The Adobe Create Now World Tour is about to kick off and if you want to learn about all the latest and greatest with Adobe® Photoshop® CC, Illustrator® CC, InDesign® CC, Dreamweaver® CC, After Effects® CC, Premiere® Pro CC, and much more, I highly suggest you attend one of these free events.

In truth, if you just plain want to get inspired, I’d suggest going.  The events are free and you’ll get to learn from the pros.

Here are the dates and how to register for an event near you:

http://adobe.ly/CCWorldTour

Oh, and if you’re thinking, “Yeah, I’ll do that later when I have time”, the first event is on September 19th, 2013, so don’t wait too long.

 

Cross-Browser Responsive Content with CSS Regions

In a recent post I described how you could use CSS Regions to easily create responsive content. One of the limitations mentioned though was that CSS Regions at present are natively supported only on Webkit nightly and Chrome Canary and Chrome Stable.

There is however the CSS Regions polyfill code up on GitHub to consider. I looked at that last year and it was a bit slow and problematic at the time.  It turns out though that there have been several improvements to the code in the interim and it works quite nicely and is quite performant now.  Let’s take a look…

Check out this Pen!

You can’t see it when the Codpen has been embedded in this post, but I’ve included cssregions.min.js in the JS tab.

Take a look at the CSS Tab.  Notice the new -adobe-flow-into and -adobe-flow-from CSS attributes:

#source {
  -adobe-flow-into: main-thread;
  -webkit-flow-into: main-thread;
  flow-into: main-thread;
}

.region {
  -adobe-flow-from: main-thread; 
  -webkit-fow-from: main-thread;
  flow-from: main-thread;
}

These new attributes are recognized by the polyfill.  They work the same as the -webkit-flow-into/from and flow-into/from attributes.  The polyfill code will detect and use any native implementations that exist so you don’t have to do anything special.  Everything should work exactly as it did before on WebKit and Chrome, but now it will also work on iOS4+, Android 4+, Firefox, Safari 5+, and Opera.   That’s right, mobile too!

Go ahead and view this post with the other browsers and devices listed above. The Codepen demo should work just fine (let me know if it doesn’t!).

Enjoy!

 

Responsive Content Using CSS Regions

Screen Shot 2013-08-16 at 2.11.43 PM

CSS Regions allow content to flow across multiple areas of the display, called regions.  The beauty of CSS Regions is that you can separate the content from the layout.  And with that comes the ability to create some responsive content very easily.

Update August 29, 2013: 

I have a new post describing Cross-Browser Responsive Content Using CSS Regions.  I describe there how you can do exactly what is described here, but also use the CSS Regions polyfill to get CSS Regions support on many other browser and devices.

The flow-into CSS property can be used to prevent some content from displaying normally, but rather have its display redirected to a named flow:

#source {
   -webkit-flow-into: main-thread;
   flow-into: main-thread;
}

Then we can specify our layout, and tell the layout where to get its content from using the flow-from CSS property:

.region {
   -webkit-flow-from: main-thread;
   flow-from: main-thread;
}

In the case above, all of the content in the element with ID source will actually get rendered into the layout specified by all of the elements of class region.

Let’s take a look at an example.  First, make sure you are viewing this with either a Webkit nightly, Chrome Canary, or an up-to-date Chrome Stable.  Then, if you haven’t already done so, you need to enable CSS regions by navigating to chrome://flags and clicking on “Enable” under Enable experimental Webkit features.  You’ll then need to relaunch the browser.

Check out this Pen!

I tried to keep the example as simple as possible.  Basically, we have a DIV whose ID is source.  That DIV contains all of the text to be displayed.  But, since we have the following in the CSS:

#source {
 -webkit-flow-into: main-thread;
 flow-into: main-thread;
}

All of the text in that DIV gets redirected into the named flow called main-thread and doesn’t display in its original DIV.

We also have several DIVs, all of class region.  In the CSS we have:

.region {
 width: 150px;
 height: 150px;
 float: left;
 margin: 10px 5px;
 -webkit-flow-from: main-thread;
 flow-from: main-thread;
}

All of those region DIVs will get their content from the main-thread named flow.  And, since they are all float: left, the DIVs will automatically relayout as you adjust the width of the window.

Finally, I stuck an image in the middle of it just for the fun of it, to show that your regions can be easily interspersed with other content.

Try that now. Make this window narrower and wider, and you should see the text automatically adjust.

Now think how much work would be involved in getting this simple example working without CSS Regions.

You can find out more about CSS Regions here.