Sunday, May 6, 2012

Prototyping challenge: Make a web-based 3D modeling toy

I'm rather obsessed with user generated content, particularly art tools.  Recently, I had a wonderful experience with Realm of the Mad God.  Alex Carobus added in a simple pixel editor that allowed anyone to create sprites that might be used in the game.  Very rapidly, players created thousands of truly delightful pieces of art.

Inspired by this, I set a design challenge for myself.
  • 3D in a browsers. What is an easy-to-use 3D modeling tool that lives in the browser?
  • Unique style:  I want the output to be instantly recognizable as being created in this toy.  That means radically constraining the tools.  Instead, I was particularly inspired by the extruded 3D style of Land-a Panda. 

  • Minimalism: Are there any ways of simplifying 3D modeling? What is the pixel editor equivalent of a 3D modeling tool?
  • Professional results:  Can we build something where you look at the results and think "Wow, that is really nice."  Think of it as the Instagram effect. I'm particularly targeting casual games, but I suspect if that is nailed, people will find all sorts of uses for the toy. 
What I'm avoiding:
  • No copying an existing tool.  Sure there are well established paths for 3D modeling or vector editing, but that is too easy.  Lets go back to the design roots of these complex monstrosities and build up something elegant and different. 
  • No voxels: I don't want to use voxels.  Minecraft already does this so let's push in a wacky new direction. 
The closest I've found that fits these constraints is the amazing TinkerCad, which is a simplified solid modeling tool.  It is very nice, but only really ticks the first checkbox.

Here's what I've come up with.  If anyone find the idea curious enough and wants to build a prototype over a few weekends, I'm happy to collaborate.  This wacky, broken and experimental.  But what is the fun in sharing only perfect ideas?

Model Toy


Model Toy: An easy to use drawing and modeling tool for making stylized objects

Model Toy is a 'back to the roots' effort that asks if you can make a modeling tool by only manipulating vertices on simple curves. The tool is made of several basic elements
  • Grid-based drawing plane: All drawing occurs on a plane.  This can feel more like a 2D tools than a 3D tool. 
  • Shapes:  The key primitive is a unique extruded vector shape defined by 4 points on a plane. 99% of the time, the artist is moving around vertices. 
  • Shape Palette:  A list of available primitive shapes. 
  • Shape Properties:  List of the current shape's color, extrusion, etc. 

Shapes

The heart of the tool are these odd 2D path-based primitives that Pete Blois and I have been experimenting with.  You can play with an example of it here: http://apps.blois.us/Drawing
  • The shape is a 2D vector composed of 3 to 4 vertices. 
  • Each vertex is either a rounded corner, half rounded or straight corner. 
  • Vertices only snap point on the grid. 
  • The shape can be extruded and beveled. 
These actually came out of a lot of different experiments and I realized something really obvious.
  • Engineers tend to make art primitives that have lots of knobs and widgets...they are highly parametric objects with a complex interface.  
  • Yet, many artists don't necessarily think in terms of complex objects.  Instead, they use simple  things that are easily manipulated and then repeat the same tweaking action thousands of times until the composite result is interesting.  There are no explicit 'rotation' or 'scale' operation when painting.  Yet the results are still impressive.  
  • So this design preferences 'tweaking thousands of times' over 'a complex object where you set variables once'.

Basic move, scale and translate operations

One interesting aspect of these primitives is that they don't have an explicit scale, rotation or translation matrix for the user to manipulate.  Instead, all those operations are performed by moving vertices around. That's all you really do in this tool...move vertices about.
  • Move shape: Click on a shape to select it. Drag on the body to move it around.  This moves all vertices together.  Note that all vertices always snap to the grid. 
  • Deformation: You can deform a primitive by moving its vertices in a 3D plane. Drag on the square surrounded a vertex to move it to a new grid point. 
  • Rotate: To rotate, move vertices one by one until the new shape looks rotated.  This is not true rotation since the snapping to the grid will not allow true rotation.   However, the result will look rotated and that is all that matters in art.  This works surprisingly well. 

There are big limits on the shapes

We could allow thousand of these objects on the screen.  But instead I'm inspired by the elegance of low resolution pixel art where beauty comes from working within limitations. 
  • All vertices are constrained to a 16x16 square grid.  This allows for easy selection of vertices and accurate adjoining of shapes. 
  • There are only 32 shapes in any one model.  This encourages the artist to create elegant compositions. 
  • Each shape is one of 16 colors in a fixed palette. 

Shape Toolbar


There are four basic shapes you can create with this method.  Click one of the primitive button on the toolbar and the shape is added to the scene.
  • Circle: 4 rounded vertices
  • Rectangle: 4 straight vertices
  • Half Circle: 3 vertices: 1 curved and 2 half curve / half straight
  • Triangle: 3 straight vertices 
Example shapes that can be created by moving vertices about on grid

    One system for defining hidden control handles

    The follow is one method of getting the desired curves using bezier handles. Straight corners are a trivia case, but round and half round need to be tweaked to allow for aesthetically pleasing circular geometries.

    • For round corners, handles are defined only by adjacent vertices (vertex 2 and 3 are adjacent to 1)
    • Handles are parallel to the line segment ‘a’
    • Length of handle is proportionate to segment ‘a’  (Note that the .27 in the diagram is a value that results in 4 round corners arranged in a square yields a perfect circle.  There is likely a mathematical means of deriving this as well, but that is beyond me. :-) 


    • For half round, half corner points, calculating the normal based off the points adjacent to vertex 1 (in the picture above) results in a bowed out shape.
    • Instead, mirror point 2 across the line segment A. This creates a new ‘Fake A’ that goes in the correct direction.
    • The new curve handle for point 1 is now parallel and proportionate to ‘Fake A’

    What this toy lacks

    • 2D scale and Rotation: With such simple primitives that are easily rearranged, we don’t need these operations.
    • Full color picker: You can’t define arbitrary colors
    • Layers and grouping: With 32 shapes, a shape list is the layer list
    • Lines: There is only the shape color. Later on, we can have effects that apply to the object as a whole.
    • Empty shapes: Shapes always have a fill color.

    Extending to 3D

    To the left is the side view palette.  This is a bit like a layer palette in photoshop, but it also lets you control Z-depth.  This is a bit geeky and isn't my favorite part of the design, but worth trying.

    • Dragging on the body of the shape moves it left or right.  This is changing the depth of the object. 
    • Dragging on the left side of the shape extrude backwards. This snaps to the grid. 
    • Dragging on the right side extrudes forwards. This snaps to the grid. 
    • The profile of the shape shows its bevel. 

    Other shape Properties


    You can select a shape and edit its properties.
    • Color: Click a shape, click a color and the shape becomes that color.
    • Bevel:  Select the bevel for the object.  No bevel, rounded corners, dome, flat bevel
    • Extrusion:  Select how far you want the object to be extruded. 

    Open questions

    • Is this expressive enough?
    • Is there a better method of expressing the 3D extrusion?
    • How might it be simplified even further?

    Near Future

    The first part of the challenge is to get a basic editor up and running. For these new drawing tools you usually need to build it and then iterate on it 5 to 10 times so that the feel of the program is solid.

    Web-based editing, saving and viewing
    The model is editable in a browser window. You can save to a database and load. You can share the model with another user and they can make a copy of it and edit their own version.

    3D view
    Once you have a 3D view you can rotate the drawing plane to see the object from from various angles.  Some experiments to try:
    • The plane always snaps back to the frontal view when you release. 
    • Alternatively if you rotate the object 90 degrees, it snap to the side view and swap the side view for the front view in the other palette.  

    Export options

    • 3D model: Exports a static 3D model for import into something like Maya, 3DS or Unity. 
    • Bitmap: Export as a series of X (64?) images rotated around a center point. Includes Alpha

    Far future

    Shader sets
    Users can load in different shader sets as alternates to the base 16 colors. For example, there is a wood set that has different types and tones of wood. Or there is a metal set that has pitted bronze, steel and copper.

    Post processing and Lighting Presets
    You can apply a variety of preset post processing filters much like Instagram. Honestly this is where the magic occurs. The idea is that these are incredibly high quality professional filters that give your simple model a distinct style.
    • Outline: Add an outline to the image so that it looks like Land-a-panda. 
    • Pop art: Dot shading.
    • Sepia: Grainy, old timey image
    States
    Define states for each model with each state have a different configuration of the 32 shapes.  For example, you could have a walk state and an attack state for a character.

    Now if you bundle these states into templates, you could provide users with a 'character template' that they can fill out to their heart's content to create a thousand unique characters that all 'work' the same.

    Animations between states
    Allow for tweening animations between states.  Add ease in and ease out for basic timing.

    Conclusion

    This odd art toy is not a perfect tool.  Having made art for a few decades now, I'm not sure there is such at thing.  Instead it is series of constraints.  The theory is that these constraints will yield interesting art when placed in the hands of motivated artists.  We've seen this happen before.  Vector art is a style that emerged from the limits and strengths of printing technology.  Pixel art emerged from the constraints of early computer displays.  There is an exuberant creativity within carefully chosen walls. Is it possible to artificially foster that?

    Mostly I wanted to share these ideas.   For the folks that love an oddball project, this might be fun to play around with for a weekend or two.  It is certainly a way to learn about curves, 3D extrusions (and the exquisite pain of iterating on an artist-centric UI.)  I'd be delighted to give feedback and try out prototypes if any emerge.

    Long term if the basics works out, I could see making an entire professionally polished game in this art style with every single character, wall, door and tree built out of these shapes. This is the real test. Once you get artist trying their hardest to build real things with a new art tool, a feedback loop is born.  The artist asks for tiny yet critical features you could have never imagined.  After a few dozen iterations, the simple odd tool begins enabling amazing artists to create a certain kind of masterpiece.

    take care,
    Danc.

    Prototypes!

    In order to keep all the learning going on in one spot, here are the prototypes that folks have made so far and feedback to each:

    Pete Blois's Model Toy - Iteration 3
    http://apps.blois.us/Drawing

    This was the first prototype Pete and I iterated on and got the basic primitives working.

    Jeiel Aranal's Model Toy - Iteration 1
    http://subjectnerdagreement.com/modelingtoy/

    This one was done in Unity and has manual control handles and some extrusion. Thoughts here:
    • Drag to move shape: The ability to click on a shape and drag it on the plane will make the tool much easier to use. (You can put rotate the view on right press or by dragging on the empty canvas.)
    • Auto-control handles: One of the neat things about the little 4 point vector objects is that the control handles are automated and not actually visible to the user. The intent is that every time you move a vertex, you look at the adjacent vertices and then calculate the length and orientation of the handles. This really simplifies the use of the tool since many users find manual control handles fiddly. (Though you did a good job putting them in!)
    • Hit region on handles: In the current build, the hit region is the circular vertex. If you use the rectangular region behind the vertex, it will be much easier to grab the vertex.
    • Mouse over: Outlining / highlighting the object on mouse over and showing the vertices makes it much clearer what you are about to manipulate.
    • Ctrl or Alt drag to duplicate: This is a classic short cut that makes it much easier to make complex objects.  Works when combined with 'Drag to move shape'. 
    • Slightly tilted drawing plane: A more complex tweak is to make the drawing plane tilted so that you are always drawing in 3D space. Since everything is still on a grid, it should be possible to still treat it as primarily a 2D drawing surface. This does require that the drawing plane be aligned with the face of each selected object.
    Mikko Mononen's Model Toy - Iteration 1
    https://tinkercad.com/sketch/curve/

    • A lovely testbed for the 2D shapes.  It is clear that there is something off with the control handle behavior. 
    • Maybe adjust the control handles independently since currently they are completely symmetrical. Perhaps bisecting A in some manner may give a better value for each handle
    • The whole thing starts feeling much better if you can drag directly on the shapes themselves to move them around the 2D drawing plane.
    • Same thought as above on the tilted drawing plane. 

    14 comments:

    1. Where's the link?! I want to try this out, too!

      ReplyDelete
    2. Cough: http://brashmonkey.com/ Spriter will eventually be set up for deformations. Not sure if they want to go full 3d though.

      ReplyDelete
    3. You make pretty good posts, don'tCHA

      ReplyDelete
    4. I'm pretty sure Spriter isn't going to be 3D. I think it's also not supposed to have actual drawing features, but it's just an animation tool. That may indeed include deformation, so I suppose if you wanted to, you could make a bunch of basic shapes and then use Spriter to use the basic shapes to build more complex things. Might be interesting. Not 3D though.

      Also, can't wait to try Model Toy. Have you checked out CraftStudio http://craftstud.io/ ? They have a sort of cuboid-based 3D modeling tool. I would describe it as a tool that lets you make models like the creatures in Minecraft. Not restricted to axis-aligned cubes like the game itself, but freely placeable and orientable cuboids.

      ReplyDelete
    5. Okay, so, if I understand correctly, the user is just defining these curves in 2D on the xy plane, and then those curves are extruded along z according to the bevel option? It sounds like you were thinking the editor itself would just be 2D, and then it would export to a 3D preview when you're done, rather than actually working in 3D?

      I would think that an actual side view visualization of the model would be really important for getting a feel for the volume and silhouette of your creation. I want to see what the extrusions look like as I'm making them. Perhaps you could integrate a side-view visualization with the depth sliders somehow. Maybe something as simple as, an object's depth slider pops up on the side view when it's selected, or scaling handles appear on either side of it. Of course, this gets a little more complicated with objects obscuring one another...

      An option for mirrored extrusions might also be good. So, say, if I have a circle curve that takes up the entire workspace, if mirror extrusion is on you'd end up with a sphere, and if it's off you'd end up with a vertically standing hemisphere with twice the diameter.

      Also, the first thing I thought of was that, if you don't have an active 3D view, you should at lest have an option to switch the main view between front and back. That should be as simple as just swapping the Z order, right?

      ReplyDelete
    6. I made a quick test to see how the curve algorithm could work, I took the liberty to add few extra shapes too (I like the triangle teardrop):
      https://tinkercad.com/sketch/curve/

      The half circle feels a bit odd, the rules work well when the apex point is "within" the straight segment. The tangent calculation for the round shape feels ok-ish, I think the length of the tangent should be somewhat proportional to the location of V1 to V2-V3. I also wish the shape would stay more round when squeezed, I'll see if I can find a bit better tangent rules.

      Based on our experience with Tinkercad, people understand 3D really quickly if your modeler has good mental model and natural mapping (i.e. which way is up, where is table, etc). We've had 6-7 year old kinds do 3D modeling, and 5th graders do can crazy things [1]. The only folks that need hand holding is us over 12-years-olds :) I highly recommend not having proxy model for 3D. I think simple rotated view 3D (maybe along the lines how Windows shows the 3D window stack) would work better for depth positioning than a separate list. I would go fully in 3D and automatically orient the camera when editing a shape.

      [1] https://tinkercad.com/things/6rPZonCVcUw

      ReplyDelete
    7. Are you familiar with the work of Takeo Igarashi and the Design Interface group he heads up in Japan? He has done a lot of really impressive work in this field. Implicit modelling and simple annotations on 2D forms to create the corresponding 3D shape.

      His research site: http://www-ui.is.s.u-tokyo.ac.jp/~takeo/
      and the interface group's relevant projects:
      http://designinterface.jp/en/projects/index.html

      ReplyDelete
    8. @Mikko Nice prototype!

      Aye, there is something off on the control handles.
      - Maybe adjust the control handles independently since currently they are completely symmetrical. Perhaps bisecting A in some manner may give a better value for each handle

      - The whole thing starts feeling much better if you can drag directly on the shapes themselves to move them around the 2D drawing plane.

      - Totally agreed that this would be most interesting displayed at a slight tilt (like the 2D windows stack! :-). Then every object has a drawing plane associated with it if you select it. So the user is still just dragging in a 2D plane.

      @id Takeo Igarashi's work on Teddy was hugely inspirational too me early on. I always thought there was some magic in the inflation-based extrusion. (I was going to add a note on that to the blog post, but thought it was getting too long already! I should add a references section! :-)

      ReplyDelete
    9. @Mikko very nice!

      @Danc After finding the optimal set of shapes, you should really think about naming this concept, shapeset or both (maybe after yourself to live 4ever). Maybe something like Dancshapes or something. I want this to be a THING.

      ReplyDelete
    10. Updated my test. Now the tangents are scaled based on how close they are to the end points, this works pretty well. The half circle fake tangents are now calculated so that the direction is from the center of A to vertex 2.

      I added button to toggle snap on/off to allow better test discontinuities in the tangent calculations.

      Whilst tinkering with this my old pet hate for beziers surfaced :) I've worked with the curves for about years now, and I still find it frustrating to discover the right shape. I once got a great lesson from the guys from underware.nl on how to organize your work for maximum bezier power (i.e. use axis aligned handles), but that is truck load on info and not easily discoverable either.

      I made a quick attempt at generating better behaving tangents, but it seems it created new set of problems. Anyways, you can test it by pressing the Tan2 button.

      @daniel, have you thought about adding more structure to the shapes? I made another quick test (no demo, sorry) where the round shape consisted of center and four handles. The tangents would always align perpendicular to the direction to center. It might not be the most expressive shape, but it was fun to discover the shapes, since you estimate better how the shape would change.

      ReplyDelete
      Replies
      1. @Mikko

        Nicely updated. The smoothness of the non-gridded version is rather sexy.

        Another tweak I noticed looking at Pete Blois's prototype (now linked in the post) is that he does a little exception where two co-linear points yield a sharp corner.

        From a purist perspective this is a little odd. However, one thing we were originally playing with is categorizing the expressive range that an artist might desire...sort of building aesthetic choices into the primitives. The co-linear straight lines ended up feeling rather nice. I'm not sure if there is an algorithm that actually yields that result, but then again art always cheats. :-)

        take care,
        Danc.

        Delete
    11. I guess I'm a bit late to this party, but I started working on a webgl based modeler for Minecraft: http://www.youtube.com/watch?v=qGLZhxH-G3E - so I guess it doesn't fulfill your "no voxel"-point ;)
      However, it uses three.js and I did copy some of the code from a tutorial...
      Either way, this is the progress I made in a few hours.

      I haven't really worked on it since, partly because I am working on other projects and partly because I have more RL-quests to attend to now.

      ReplyDelete
    12. Would it be an option to have the source code of the prototype made available? Like that others may be able to build on the existing work too.

      ReplyDelete
    13. TinkerCad is closing! Next project is called Airstone.

      ReplyDelete