Thoughts about bay area wage inflation

So the recent article about CEOs conspiring to keep (those poor people) wages low made me think ....
(besides the fact that the article is not making a single mention that we are talking about the most highly paid engineers in the world - nobody would dare to write such an rticle about goldman sachs conspiring with other mega-banks to keep the comp. packages of investment bannkers low - and we are soon going to be talking about similar comps for senior engineers (ie 500K - 1M total annual comps..) )

So my simply realization was the following
 - singularity theory of exponential progress is hitting the bay area earlier than other places, more disruptive companies here, bay area's market growing disproportionally
 to the rest of the US/world
 - larger market/cap => requires/buys more brains => physical limits and people inertia cause two things
   - mega real estate bubble from rents to real estate price to commercial real estate price (aided by a restrictive residential market / cornerned commercial real estate market by a few real etsate tycoons)
   - wage comps skyrocketing - at the base of the pyramid due to real etstate/living costs at the top of the pyramid due to the supply demand imbalance
 - the first reaction by the mega employers is to do their own "cornering the marker" is bound to fail - there is no oligopoly - the circle of trust needs to keep on expanded as more and more new employers are breaking through (twitter etc) - we are seeing its unravelling now as the article points
 - with a fully transparent (linkedin driven) talent marketplace there will be an all out war for talent - which will force the typical $1M/eng in acqui-hires to become more common in steal-hires
 - the inbalanace between comps given to new from old employees (new getting 2-4X of what older hires get) will force more talent liquidity as well as startups that will make the last "secret" part of the talent marker visible (what people are actually making)
 - that will feed further the push on salary increases across the pyramid of employees, the total impact to the profitability of the companies will be felt rather strongly
 - the realization that r&d costs by a company are not fixed costs but are (almost like) variable costs, you need to keep on shelling out $1M/eng every 2-3 years to replenish your highly liquid talent - without them your technology advance is immediately reversed.
 - the only escape hatch of this localized inflationary pressure that bay area will be feeling is to escape out of the bay area.
 - it is questionable whether/when the friction do the problems above will counter-balance the increased advantages that bay area has due increased critical mass of the disruptive innovation key ingredients
 - most probably the first step will be that companies will increase their telecommute programms with people living in surrounding areas (just like od did) with people from southern CA, washington, utah etc..
 - younger companies - being the most unable to compete in this "war for talent" will be the fisrt ones to be forced to change into more/majority global employee teams
 - younger companies are going to be the big fish companies in 2-5 years - bringing these trends to the mainstream tech companies of the bay area.
 - companies will become global ready - people will be more comfortable running meetings remote - having their key contributors being remore as opposed to local
 - startups and technologies will further fill the gaps that cause innefficiencies for remote people, from addressing collaboration issues, temporary housing isses, payrol tax issues

 => the above transformation will happen first - the next step will be to accept the full concept of truly global (as opposed to US only) on-demand online worker as the one pitched by OD

Another thought. In the past using bay area as a test market carried a huge caveat - "the rest of the world isn't like us"...   However, as bay area becomes an increasing portion of the $s of the world certain startups that are not targeting people but $s can safely focus just here in the bay area and expect to get the lion share of the available market. Even for those companies that target consumers... you can make the case that bay area gives you a future-time window ab testing of what the rest of the world will be 2-5 yrs later right now (and that time-delay will tend to be come shorter and shorter in the future)

Job benefits laundry list

Random list of all aspects of someone's job that may have a cash-equivalent effect  on their assesment of the job's value:
Maybe worthwile as a tool for someone to assess the value they get from their current job


- salary
- annual bonus
  - what portion is based on company performance vs personal performance
  - what % of the total bonus was given the last 3 yrs to people
- annual salary growth policy (what was the salary growth on avg in the eng team last year?)
- expense policy
- signing bonus
  - does the company pay for moving in expenses
- does the company pay for my bus/train tickets
- does it have a free daily meal? how often does the company pay for free meals
- does the company pay for my daily breakfast?
- does the company pay for mydaily coffee/snacks/junk food
- does the company pay for a new laptop/better than what I have? What is its value? How often do I get to upgrade it?
   (if better than mine - e.g. a $4K Macbook Pro...that I can replace every 2 yrs that is incremental
   value compared to a 2yr old on avg owned 1.5K air  of ~1K/yr. 
   I get 50% of that extra value... the company gets the other 50% 
  (so ... even for something like that I can create a meaningful monetary equivalent…)
- How many big displays do I get on my desk
- do I get a good ergonomic chair? anyother better/worse than avg workstation equipment?
- does the company pay for extra broadband internet connectivity mifi/extra home bw?
- does the company pay portion of my cell bill - since I may on pager duty?
- does the company pay for annual conferences? what is typical budget? trip? ticket? How many days off do I get for that?
- Do I get monthly odesk/elance/mturk budget to allow myself to better scale/outsource my work to others?
- open source - does the company feels comfortable open-sourcing generic tools/libraries that it built?
- how many aws/heroku etc resources/budget do I get for my work projects? toy projects?
- do I get a paid/premium github/dropbox/yyyy account?
- do I get to work from home once in a while? once a week?
- do the people that work in the office know how to work with remote people (hangouts, online docs/prese.. or having meeting means have to be in the office?
- add the commute time benefits to that (once a week => 52 hrs => 2.5% of 2K annual hrs
- when calculating commute costs/benefits generate two pieces => one that acts
  as a salary discount/addition (hourly rate x...) and another that captures the actual commute cost
   (1hr => 1/5th of tank > $20-25 per day
- to how many meetings do I  expect to be. (ask for the calendar of collueges to see coverage....
  Assume 25% avg if more than that you are less efficient => apply discount for lack of job satisfaction
- do I get to work directly with famous people
- how famous/well known is the company I will work for
- how fancy is my title
- do Iget my credits on the film? (my photo/name on the website?) ? 
- do I get to talk to conferences/workshop etc about what I do?
- How many interviews and time do I need to spend to get hired?
- Do I need to change my work attire?
- Can I bring my dog to work?
- Will others bring annoying pets to work?
- Don't I have a private office
- Arent we working in an open-office env
- What is the avg noise of the office?
- Is there free parking place? How hard is it to park?
- Can i bring my bike in the office?
- Are there electric car outlets nearby
- How far away is the train station, frequency of the stop, bullet train stop etc
- How bad is my commute for days that I have to be in the office during the traffic peak time?
- how often do I have to be in the office such times?
- stock options
  - do you stock options (find out strike price, current common share value (409A), cliff
  - how future/follow on grants are structured
  - what is the strike price
  - what is the cliff/vesting
  - what is the usual plan for future add-tional options
  - how often does the 409A get updated
  - does the company have clauses the prohibit share exchange in secondary markets
  - does the company treats its employees as shareholders - and shares with them regular financials, cap table break up - vacation days per year
  typical job will allow you to take 15 days of vaca+sick (15x8 hrs = 120 out of 2K annual hours)
  +/- of that amount add in the comp
- commute time to work
  assume that avg work implies 1hr commute. a 2 hr commute => 1 extra hr per day => 10% more time
    add a 10% discount (to the job's salary?)
 - Do I get to use Uber every now and then?
 - Does the company offer in-office dry-cleaner etc pickup (purple tie)
 - Do I get any taskrabbit monthly allowance to deal more efficiently with personal logistics
 - internationals
 - does the company pay for my H1? green card?
 - does it have experience doing so? When does my application can start? Can I control the timetable?
 - Is the company willing to pay do the "fast track" path?
 - will the company be ok with me spending some time in my home country - working remotely - as a way to provide a longer vaction to my family?
 - does the company fores me to speak English even with my homo-ethnic colleagues?

Normalizing Job Offers

A friend that seemed to be enjoying job interviewing way too much (spent more than 6 months each time courting with companies during his two job hunting phases since I get to know him 3yrs ago) had created a little calculator to allow him to compare often rather dis-similar job offers.

The problem here is that there is an underlying assumption that companies would benefit by not becoming easily comparable on a simple quantitative yardstick to other companies - they somehow benefit from offering a unique product that is its own little monopoly - ie a product that is not easily comparable to other products - the marketing pitch around the job is always trying to drive home that uniqueness. In some way they try to make applicants think that money should be the last thing they should be thinking when they are applying to work for this company.
But thats inefficient and wrong and against the current trends of transparency, quantification, objective measurements and optimization, data science etc.

The silly thing is that companies often do have the best cash-equivalent offer - even if they don't give the highest salary. The vast majority of the job benefits can be quantified so as we can obtain their $-equivalent, whether it is free meals, working from home, giving shares instead of stock options or  a low strike price. Its just a bit  hard and we don't have good tools to do it.

The other side of this problem is that the lack of transparency  makes collecting this info hard -
especially for a new person - and not only that but not transparent/official rules that the company treats in the same way as cash compensation can subtly change - with a new manager, or a not so positive board meeting - altering the unwritten contract between the employees and employer.

So I think this is wrong and there is a way to fix it.
It is hard to fix it for the outsiders - they get exposed to a tiny part of what the company truly - but it can be fixed for the insiders. The insiders have all the above information and they can tryly assess the complete compensation package that the company actually offers. Or putting in the inverse way they can easily determine the price that someone has to beat to make an employee consider their offer.

The question that no-one in the job market dares to ask/answer

Everyone in the job market at some point or another is going to discuss an offer.
An offer is described in the context of a prospective hire.
The discussion around it is trying to pinpoint the right answer to the question:
 "for how much am I willing to join your company"

I think a more interesting question is instead a question that people don't ask (and would probably feel even more uncomfortable to answer)
 "for how much am I willing to leave my current company"

This "pivoting" of the question has the following interesting characteristics
 - It is only valid for people that currently do have a job.
 Thats ok - most prospective employers would rather not deal with people that never had a job or people that are laid off. They would be interested in talking to people that quit their jobs but people quite their jobs typically after having found what job to do next
 - It is also valid for people that are working on something - even though they don't have a job
 For any wannabe enterpreuneir  / not really getting paid startup coder, founder , cofounder or cofounding employee all these people that may be working but they don't really have a job this question still makes sense : it means for how much I am will move from my self-employed state to become (again) a hired employee.
 - It is agnostic about the new job : an offer to a person encapsulates two elements - the cost/benefit/opportunity loss of someone leaving a prior company together with that obtained by joining a new one.. Contrary this questions only capture one side of the equation and that makes a more useful/primitive knowledge piece.
 - It is an always valid to ask question: I can always ask and answer this question I don't have to be in job hunting mood. The answer to this question is the barometer of how much I like/I value/I benefit to staying to my company.
 - It is not a secret: the offers, how much you are being paid etc are all secrets protected with legal clauses in the employement contracts of employees. However, answering this question doesn't seem to betray any confidential information - even though in reality it does expose the true total $-ized benefit of the employee by their employer
 - People wouldn't dare to say it. In spite if the confidentiality clauses - people do talk about their salaries/offers etc.. Do they really open up to answer this question - hardly ever... It requires a significant sense of personal honesty to answer this accurately (and braveness to answer it openly)

It would really be cool if we could get people to think and answer this question....

Charging for free services that have computational needs

I love the way wolphram alpha does it....

Note that I have already done the search once - and got the right result ($17T approximately).
It seems that the service recalculates as opposed to serving cached results.
The service gave answers to several questions that seemed to require quite some time
(google+apple) gave me lots of outputs  - when I added every single mega sw tech company since the PC era that I could think of....

it then gave much fewer data points - still taking some time to answer.

So 

- point 1: the idea of allowing free usage but when your computation requires more than X - instead of , kicking the user out, penalizing everyone, making the user wait for ever, you provide the option to pay and use more resources...  

It reminds me the chess AI idea of greg to allow chess players to use an chess-player-assist program to play against their opponent (who also uses the same program...)  - and either people can pay to get more resources - run the alg at diff depth or both be limited by the # of funds that are being bet on the game - they can use the funds to get help - essentially reducing their potential earnout

 - point 2 : I really want to see the Market Cap of companies created the last 20-30 years as a % of the total over time since 1900 - do the same using the HQ address location etc... (e.g. bay area generating 2+Trillion dollars of market cap growth in 10-20yrs). 

The hardest challenge : finding jobs for people to do

An odd friend of mine told me several years as we were discussing about labor marketplaces - and whether the future holds a work-supply or work-demand challenge - that the problem is actually quite different.

He gave me a challenge :

I can give you as much money as you want (he meant it) to create jobs for them in the OD marketplace - but with one simple condition. The workers that get hired should get the impression that these are real jobs. You will soon realize that solving that problem in a scalable way is at least as hard as finding money in a scalable way.
The discussion continued with me saying things like

 - I will create jobs to translate every single wikipedia page in every other language
 - I will hire people to write ficticious jobs and then hire people to do them
 - I would post all generated issues in github as projects
etc..

He had a (possibly questionable) way to dismiss all of them based on the following logic :
 - the jobs produce no actual value in which case - people soon or later figure out that their jobs are pointless - at which point he would either say I violate the assumption above or he would say that the effort that would need to be added to confirm that a pointless job is actually well done is impossible or not scalable
 - the jobs produce actual value - in that case he would try to prove that the amount of value would decline over time - after million tasks/million people the value is too small - so he would use the prior argument  (pointless jobs) or that I have no scalable model to expand the solution for the society as a whole.

I think at the background of my odd-friend mind lied the idea that we are moving towards a matrix-like dystopia - where necessary resources (food,shelter,entertainment,health) are available for all - practically for free - as a result of the work of a tiny% of the earth's population (the "smarter" 1-10% of us) - still the only way to keep the planet in balance is to make the vast majority of people maintain the illusion that what they do for living is an important/necessary part of the value generation and attempt to attribute some reward <-> effort model to keep them do whatever they do - otherwise everybody wakes  up and realizes that they are in a sham world build by others and placed in front of their own eyes - and the system/society unfolds.

What brought that chain of thought - reading the title   Why we should give free money to everyone of this HN article... The article is on a different topic - but is still relevant - welfare / workfare are incarnations of the same problem as the one that my friend talks about.

How often do I google

I wanted to find out the distribution over the course of a day of a typical person's google searches.

A friend in the office said that I should look at real data ... I guess the only data I have was mine. I looked at my chrome web history - chrome doesn't have timestamps there...

Ok.. after a bit of digging I found it :

For a Mac user its

ithaca> cp ~/Library/Application Support/Google/Chrome/Profile 3/History /tmp

ithaca> sqlite3 /tmp/History

sqlite> select avg(cnt) from (

  select ddate,count(*) cnt from (

    select 

      date(last_visit_time/1000000-11644473600,'unixepoch','localtime') as ddate,       time(last_visit_time/1000000-11644473600,'unixepoch','localtime') as dtime 

    from urls 

    where 

      url like '%google.com/sea%' 

    order by 1 desc 

  ) a 

  group by ddate 

) b;

13.0923076923077

Interesting things there:

  There is a history file for each profile of yours as you would expect

  The history file is an sqlite db file! A very nice surprise

  The timestamps aren't unix epoch based - they start somewhere in the 17th century for some reason - so you need to do the adjustment (SO gave the right offset)

  I still can't believe that browsers treat web history as something disposable - no clue why mine is truncated 2-3 months ago. 

  A tool that is syncing the history file daily/weekly with a more persistent DB (from all my profiles) may be worthwhile

  On Average I do 13 searches per day (I was expecting that to be more given how frequently I use the chrome omni-bar..

  Reference to the previous post - Google is making at least $30/mo from me from my search activity..... 

If I knew R better at this point I would just show the histogram of these timestamps but given that I mostly use as a command line calculator ..... this this will take me more than a few minutes...