Category Archives: Tech

Intro to Computer Vision

I’m new to computer vision and a lot of the basic concepts are very interesting. As an iOS developer, my interests comes from using CoreML & Apple’s Vision in apps to improve the user experience.

Two common tasks are classification and object detection. Classification allows you to detect dominant objects present in an image. For example, classification can tell you that photo is probably of a car.

Object detection is much more difficult since it not only recognizes what objects are present, but also detects where they are in the image. This means that object detection can tell you that there is probably a car within these bounds of the image.

What’s important is that the machine learning model runs in an acceptable amount of time. Either asynchronous in the background or in real time. Apple provides a listing of sample models for classification at https://developer.apple.com/machine-learning/.

For real time object detection, TinyYOLO is an option, even if the frame rate is not near 60 fps today. Other real time detection models like YOLO or R-CNN are not going to provide a sufficient experience on mobile devices today.

One other interesting thing I came across is the PASCAL Visual Object Classes (VOC). These are common objects used for benchmarking object classification.

For 2012, the twenty object classes that have been selected were:

Person: person Animal: bird, cat, cow, dog, horse, sheep
Vehicle: aeroplane, bicycle, boat, bus, car, motorbike, train
Indoor: bottle, chair, dining table, potted plant, sofa, tv/monitor

These are common objects used to train classification models.

Computer vision used with machine learning has a tremendous amount of potential. Whether used with AR or other use cases, they can provide a compelling user experience beyond Not Hotdog.

Rob Pike’s dream setup

I recently came across a short article on Uses This.

Rob Pike talks about his setup. His response for his dream setup & having stateless devices really intrigued me.

Twenty years ago, you expected a phone to be provided everywhere you went, and that phone worked the same everywhere. At a friend’s house, or a restaurant, or a hotel, or a pay phone, you could pick up the receiver and make a call. You didn’t carry a phone around with you; phones were part of the infrastructure.

– Rob Pike
https://usesthis.com/interviews/rob.pike/

This is hard to imagine today or in the future for computers. A world where you didn’t have to carry around a mobile phone. You simply used the mobile phone wherever you went. For many reasons, what Rob talks about for phones (applied to computers) doesn’t seem likely.

But it’s nice to imagine a world where you didn’t have to update your phone every couple of years. You could rely on a device at home or at work to pick up where you left off, without having to lug something expensive around and keep it charged.

ARKit Impressions

I’ve been working with ARKit recently. I am planning on releasing an AR basketball game when iOS 11 is released.

Here are misc thoughts about working with ARKit:

It’s hard to find answers to common questions about doing simple things in ARKit. Searching for SceneKit yields slightly more results, but even that is sparse. The Apple developer SceneKit & ARKit forums don’t appear to have much activity either. So it’s up to StackOverflow & random Internet blog posts
Working with ARKit means working with SceneKit. SceneKit is Apple’s framework to make working with 3D assets easier for developers. Working with SceneKit & 3D is something that I’m new to. A lot of the math around position, orientation, euler angles, transforms, etc. can get complex fast when it involves matrix transforms and quaternions.
It’s really hard to find assets for DAE/collada. The DAE format is meant to be interchange format for various 3D software to communicate with each other. The reality is that exporting to DAE or converting from one format to DAE is a crapshoot. I’ve used Blender briefly to look at 3d assets, but digging into 3D modeling is a huge time sink for some one looking to get involved in ARKit. I wish there was an online store that focused on selling low poly (<10K), DAE files.
Related, working with 3D assets as someone new to 3D assets is very frustrating. The concept of bounds vs scaling as they relate to importing into your SceneKit scene was very challenging (with the 3D model that I imported). If you have your own in-house or contracted 3D modeler, you should get 3D assets that work well with SceneKit, but I had countless issues with off the shelf 3D models & file formats.
After you’re able to import your 3D model, modeling the physics geometry can be a challenge. SceneKit allows you to import the geometry for your physics body as-is using ConcavePolyhedron, but you probably don’t want that. I had to manually recreate a basketball hoop using multiple shapes combined into a single SCNNode.
ARKit is not all powerful. The main feature that ARKit gives you is horizontal plane detection. Occlusion doesn’t come with ARKit. Expect many apps that deliver an experience reliant on a plane/surface like your desk or the floor.

ARKit is exciting, but don’t expect the world yet. Future ARKit releases & better iOS hardware should provide more compelling experiences. Today, you can expect to play with 3D models on a surface (with surface interaction) or in the air (with limited or no environment interaction).

App Strategy

On the subject of doing app planning & strategy, I recently came across this post from Rob Caraway: http://robcaraway.com/blog/index.php/2017/02/12/how-i-overcame-crippling-perfectionism-and-made-200k-on-the-saturated-app-store/

Parts of it really resonated with me. He says:

Our strategy was basically “Let’s brainstorm ideas and ship massive features and hope people want them”.

That has been my naive strategy so far. Acting as my own ideal user.

Then he talks about validating a MVP:

using “Traffic, as indicated by Google Trends”
a landing page to capture e-mails
building a prototype in a week
validating the demand for the prototype

This all seems standard or obvious when you look at it. But I can say that in reality, I have various app ideas that I think are worth making. When it comes to pick the next one, my current process might as well be rolling dice with bad odds. It’s 1000% obvious, but building a neat app with good UX in 2017 doesn’t count for much. Having a solid marketing strategy in a validated niche is significantly more important than building the best app ever.

I’m currently at a point where I’ve released 3 iOS apps. One of them has done decently and the other two are not. I have to make a decision between prioritizing developing new features for my current apps or creating a new app. For the sake of learning new iOS tools (like the camera), it’s probably better for me to work on a new app. Hopefully I can properly validate my idea before I spend months building it this time.

Tip Solver launches

This week, my future-inspired tip calc, Tip Solver, launched on the App Store. Not only does it help you calculate you tip, it also helps you solve for how much you are really tipping.

As a tip calc, it’s easy to use and hopefully sleek/easy on the eyes. A key feature of this tip calculator is that it allows you to solve for your tip %. When you adjust the total or tip, you can see how much you actually tipped right away. This comes in handy when you pay $100 instead of $95, and so on.

While it’s customary to go from top to bottom when looking a receipt (for the items ordered, tax, tip, and total lines), I felt that it was better to invert the direction (to use a bottom to top approach). The reason being that your thumbs are often better able to reach the bottom (not the top) of the device. I wanted the most common action (setting the bill amount) to be in the easy to reach thumb zone.

Please check out my app if you have a moment. It’s free (with ads) and works on iPhones & iPads. If you have any feedback, you can let me know at rexfeng@gmail.com

https://itunes.apple.com/us/app/tip-solver-premier-gratuity/id1130814051

iOS Tip Calculators

I am in the midst of wrapping up a new iOS app. Wrapping up an app includes so many things that are oftentimes overlooked when it comes to developing a mobile application. From App Store screenshots, intro video, and description text, there is a lot of room to do it well (or poorly).

My new app is a tip calculator, which is by no means a new idea. The reasons I decided to build a tip calculator was that I wanted 1.) to implement modern usability improvements and 2.) have an app that is visually attractive.

Looking at the App Store, most calculator apps adopt a heavily skeuomorphic style. There is nothing wrong with skeuomorphism (in the context of aiding usability), but I wanted to build a tip calculator that is sleek and does not resemble a calculator.

Below are screenshots from the current top search results for “tip calculator” in the App Store. They are all probably decent apps that work, but I’m posting them as a reference of what the current state of tip calculator apps looks like.

Screen Shot 2016-06-22 at 7.26.17 PM Screen Shot 2016-06-22 at 7.27.05 PM Screen Shot 2016-06-22 at 7.28.03 PM

LA Commute Visualization

This month, I chose to visualize LA commutes. The data is visualized & published here: http://xta.github.io/la_commute/

la_commute_ss

For the starting locations, I chose a mix based on highly populated areas and places of interest to me. I wanted to get a good distribution throughout the greater LA area. I realize that most people wouldn’t commute nearly two hours a day, but the sad reality is that people do have these long or even longer commutes.

For the destinations, I chose three popular, work-concentrated areas (Santa Monica, Century City, and Downtown Los Angeles). I realize that many people do not work in these 3 locations, but these locations help visualize a horizontal slice across central LA.

My workflow was running a local Ruby script multiple times a day throughout the past few weeks. I have both morning & evening commute data, but I think morning commutes are more interesting. I may be wrong, but I’m assuming that morning commutes are more consistent (people leave for work around 7 to 8am, and they leave work anywhere from 3 to 9pm).

Once I had the data, I loaded my csv file(s) into Google Sheets. With Google Sheets, I did basic sorting and aggregating of the commute times. I output my data in a specific format so that I could easily consume it with my JS code.

Loading the data into Leaflet.js markers wasn’t too bad. The hardest part was styling & displaying the commute data properly. Originally, I wanted to draw labelled lines between the different locations to the destination, but labelling lines appears to be really difficult with web map libraries. I also didn’t want to hide all the data behind a tooltip that had to be clicked.

Overall, I’m pleased with my basic workflow and the power of Leaflet.js. Collecting traffic data was the most difficult part.

Mapping library used is Leaflet.js. Map & map data are from OpenStreetMap contributors. Map tiles are from CartoDB.

iTunes Payments Dropdown Tip

A quick tip about iTunes Payments and Financial Reports in iTunes Connect.

I was trying to reconcile from iAd Revenue to iTunes Payments. It looked like there were amounts missing in the Payments page. It took me a long time to realize this, but there is a dropdown to switch between your organization payment accounts. I probably have multiple payment accounts in iTunes Connect due to updating my address.

If you have multiple iTunes Connect agreements or accounts, you may want to check for a dropdown on the Summary page. I hope this helps someone out there trying to understand this part of iTunes Connect better.

itc_pmts

Optimizing for stable tools that don’t create perpetual work

Time is an incredibly important asset.

I come from a Ruby on Rails background. The progress of Rails updates & JS frameworks has been amazing & constant. Each new Rails patch brings with it some work to stay current. It’s not Rails’s fault since there are always new features or security issues that arise. Having a well maintained framework, such as Rails, is a huge boon for the community.

With any programmer tool, you generally want to be on the current stable release (for a variety of reasons including security & bug fixes). The issue is that upgrading to the latest stable version creates a never ending stream of (hopefully small) work.

Even if you went without a framework (Rails, Django, etc.), your server is running on a suite of tools. You’ll need to keep your OS (even LTS) and most likely nginx up to date.

Perhaps you want to outsource server maintenance, so you’re using Heroku. You’ll have to keep your configs compliant with the Heroku deployment framework & best practices.

What I’m getting at is that there are so many incredible tools available to developers today. Oftentimes, these tools are free and constantly get better & faster over time.

I’m wondering if there is, or if it would be possible to create developer tools that are optimized for API stability. No more figuring how to do things the framework-way every several months. Setup once, use forever. When you’re able to minimize the present value amount of time spent maintaining a tool, you’re freeing your future self to work on higher value tasks.

RexBlog

Technology Culture