Search This Blog

Friday 22 April 2011

More detail on Apple’s iPhone Tracking Data

It seems highly likely to me that the tracking data retained by iPhones and (currently) backed up in the collections.db database is fairly innocuous.

I pulled together some Perl scripts to analyse the data and have come to the following conclusions:

  • The CellLocation Table (8000 entries) is a subset of Apple’s global cell tower location database that is cached locally. The timestamps in this only makes sense if they are part of a batch update from some other source. In my case there about 100 distinct groups of location and they don’t match where I’ve been at any level smaller than about 50km and on a timescale of days. There are some really odd entries too – one in northern Italy that I haven’t even flown within 500 km of in the past year.
  • The WiFi Location table (60000 entries) has similar characteristics. It’s got groups of AP locations that also make no sense. My table correctly has a bunch of locations in Bracknell during March but there are also more than a hundred entries much further North at the same time in a curious east <-> west line that only makes sense if there is some caching process that pre-fetches a bunch of these to improve non-GPS positioning performance. All the MAC addresses listed appear to be infrastructure (AP’s) rather than user systems so the general threat from it is pretty limited.
  • The CellLocationLocal Table looks much more like tracking data but it only has 106 entries. All of these have unique timestamps, include altitude, speed and heading information and most importantly all seem to correspond very accurately (to within a few metres) with times and places that make sense even though they claim the accuracy is only 1500m. Thinking carefully about it all of these seem to correspond with times when I had the GPS function enabled. It looks to me as if this is data that Apple might actually be likely to harvest and could be sending home in order to improve their Cell Phone location maps.

Having discussed this with some folks and after reading the online commentary I think it’s pretty safe to say that this looks like a storm in a teacup. There is certainly some tracking data in the file, but it’s privacy risks are a lot less than they seemed. Apple don’t appear to be gathering location data that includes the DeviceID or any other details that could uniquely identify the user or phone. They could be sending that back along with such an identifier but I doubt that they would – having these tables structured in this way only makes sense if the primary purpose is to enable a local cache for quick location lookups.

Ultimately I’m a bit disappointed – I’d wanted to have a reliable record of where I’d been over the last year – but on the plus side it really doesn’t look as if Apple are harvesting data that could be used to spy on me. We’ll have to se what comes out on this over the next few days but I’m not losing any sleep over this.

Wednesday 20 April 2011

iPhone Location Tracking

The subject of iPhone’s tracking their owners locations has hit the Interwebs again because Pete Warden and Alasdair Allan just pulished a rather nifty application that extracts and maps out the location logs that iOS keeps.

Their application uses the location info that is acquired using cell tower triangulation which is of dubious accuracy (about 500m for me for the most part) and seems to have something weird going on with it’s timestamps in my case, but that might just be something strange about either my iPhone or O2’s network. They get all of this from the CellLocation table within the iOS database that Apple use to manage most of the OS system configs and functions (Consolidated.db). Finding the Database is a minor challenge given the instructions provided. Extracting the data is the work of a few clicks after that.

Interestingly they don’t mention the fact that there is dramatically more and far better GPS derived data in the Location table which has far fewer entries and only seems to log data when you are actively using a GPS app. 

And there is a WiFiLocation database for those times that you have WiFi enabled that has logged about 8 times as much data as I have for Cell towers. That particular table intrigues me because it contains MAC addresses, and there appear to be lots of them (120k entries in that table). Interestingly it only records MAC addresses (along with location data and timestamps) not SSID’s which confirms one of my older assertions that SSID’s are useless for location tracking. I’m going to take a look at that table in much more detail to see whether I’ve been harvesting thousands of MAC address\Location combos silently over the past year.

It would be very interesting to know whether Apple extracts any of this data, and if so what it does with it. Kim Cameron had a lot to say about the risks of this last year when he made some fairly insightful remarks about the massive privacy holes in Apple’s Policy. At that point we were only talking about Apple gathering the user’s own device ID but I am at a loss to explain why Apple would have the phones log all of this location data if they did not intend to harvest it.

Funnily enough all three location tables also have a corresponding “Harvest” table but they are currently empty on my iPhone at least, perhaps they have plans for some future capabilities.

I’m thinking of putting something together that will allow us poor Windows users to get the data poured into a nice Map interface, Pete and Alasdair’s version is OSX only at the moment so I’m making do with Perl and Google Apps to see what data SkyNet has been collecting on me. It’s not very accurate as I noted above but it’s logged all 20 of my international trips over the past couple of months at some level.

Sunday 13 March 2011

Earthquakes and Tsunamis

I’ve been following this with a seriously high horror quotient but the following quote from CNN live reduced me to tears:

Since the initial earthquake, there have been 250 aftershocks above 5.0 and almost 50 above 6.0, CNN's Chad Meyers said.

We live in a potentially savage world but that is so afar beyond the pale that I don;t know what to say – even one of those 250 “minor” quakes would reduce this country to rubble and anarchy – and these people have had to withstand hundreds of them.

I sincerely hope it gets better now , rather than worse.

Saturday 12 March 2011

Quack Medication and Professional Ethics

I believe that the multi-billion pound\dollar alt-med "movement" is a real problem that is a shame on our society. Apart from the areas where it accidentally leverages placebo effects it does nothing positive at best, can be hugely destructive at worst and relies on fantasy and a denial of reality as fundamental principle.

As an example, the overwhelming medical research* out there indicates that Homeopathy does not, and cannot, deliver positive medical outcomes. I’m picking on them as an example but like the rest it’s a hugely profitable business based on fantasy. Fortunately, unlike some others (many herbal teas can be toxic in large doses for example), there is also zero risk of direct negative medical outcomes. The opportunity cost, however, can be fatal for those ill-informed enough to put their entire trust in it which is not a trivial consequence. As a business it’s about selling little bottles of water and sugar pills where the raw material costs of the ingredients are effectively zero. All real costs are for packaging, marketing and distribution alone. Even better they can make “stronger” products (ie more expensive ones) by (further) diluting the ingredients. Imagine how you could rake in the cash in any other business if you could persuade your customers to pay you twice as much for half the product. From a purely business perspective, that really is magic, but it needs a lot of effective marketing to convince people to pay for something that does nothing. Most of that marketing comes from effective placements in Pharmacies, because we’re all conditioned to trust our Pharmacists’ professional medical judgements.

That it exists at all may be an insult to rationality, but at some level if people want to be duped then I can’t fault a smart business for chasing that market. However a large part of its continued success stems from the fact that the public perception of its validity is massively inflated by the fact that Pharmacists appear to a casual consumer to consider them as perfectly valid medications.Now I know that many will argue that they are (usually) displayed and labelled in a way that separates them from real medications but I cannot understand how they can fail to acknowledge how weak that argument actually is.

From my perspective the fact that Pharmacists dole out Homeopathic and other “alt-med” quack cures would be like an Electrician who would happily wire up your house with wiring made from wool wrapped in paper. We don’t get duped by wool and paper pseudo-electricians of course but that’s because any fool can demonstrate that some wool wrapped in paper wont power your kettle. With medical issues we need trustworthy medical professionals to tell us whether some possible treatment will do what we need it to do. Proving medical outcomes is hard, requires expertise, and defending the rigour required by those processes in the face of the overwhelming force of the profitability of the quack medication industry is hard. Given the trust we put in Pharmacists I feel that they should be obliged to act ethically in this regard, and they don’t.

Before I go on I probably should also say that I don’t particularly trust the Pharmaceutical Industry as a whole, the companies that make all our (legal) drugs that is. It’s got its own problems, some of them immense, but for the most part their business is to make stuff that has genuine medical effects. For all the corruption that there may be there, they are accountable to rationality at some level and their products are subject to extensive scrutiny with regard to their benefits and harms. They are also profit-seeking scumbags in many cases, but they actually do deliver reality based medications that have transformed the medical outcomes of virtually everyone on the planet. They are subject to license (in most places) and have to act accordingly. 

In Ireland we have a tightly controlled consumer pharmaceutical trade. I cannot buy more than a handful of paracetamol pills at a time (for well intentioned medical reasons), codeine based OTC painkillers are now only available after talking to a Pharmacist (again arguably a valid requirement that helps limit potential abuses), and pretty much any genuine medication has to be purchased through a regulated pharmacy. We give Pharmacies this monopoly because they are rigorously regulated as professionals and that provides benefits to us because they control the distribution of many beneficial but genuinely dangerous drugs.

The downside is that, as with any other monopoly scenario, we pay many times more for common drugs here than we otherwise would. If I recall correctly from my last trip to the US I could buy many common drugs (ibuprofen, aspirin, paracetamol, famatodine) easily in bulk at costs up to 40x lower than here. And if I can buy a couple of thousand ibuprofen pills in the US for the cost of a packet of 24 here then, frankly, the bloody pharmacists should be doing that for us instead and making it obvious to us when we are about to spend €5 on a couple of Neurofen that they could give us 100 generics for €1, or whatever. And by all means be careful about selling 1000 Paracetamol to anyone stupid\depressed enough to try and down them all in one go but don’t use that as a shotgun excuse to extort 95% profit margins from the 99% of the population who aren’t going to do that.

But all of that is a minor business ethics faux pas compared to the medical ethics catastrophe that is their willingness to collaborate with the various quack pseudo medication scams out there. The sight of magic cures sold by Pharmacies and presented as if they were on a par with real medication is a shocking example of profit trumping medical ethics. How on earth can anyone trust a pharmacist's ability to provide competent advice on taking medication when they also sell magic magnetic bracelets, water potions, Flower Power pills and Mr Fruity's Frightfully Useful Unguent. They might as well have displays for Ollivander's Magic Wands, quantum dream catchers, holographic pixie dust, compressed unicorn karma, blue moon crystals and other trans-dimensional snake oils sitting by the till under a sign that says "Guaranteed Cures for all your Ailments".

Frankly in doing this they whoring out their professional reputation in order to delude the vulnerable and make a quick buck. I really find that despicable. I’m sure their businesses are complex, and making a profit is relatively hard overall so in their defence I’d actually be happy enough to let them charge me 40x the cost for a few painkillers if I knew that cost also made their business viable enough that I would then trust their medical judgement when it came to the stuff they sell. But right now there is no way that I can, and I don’t see how anyone could.

As a possible compromise perhaps Pharmacists should be obliged to treat alt-med products in the same way that they deal with codeine based products: Hide them behind the counter and interview all potential customers. After all the purpose with the Codeine rule is to protect people from their own stupidity, so if they took the time to have a chat with anyone looking for magic water pills so as to explain what they actually contained, the real benefits\risks attached and then form a judgement as to whether the sale was going to be helpful to the customer before allowing it, then they could ethically continue to sell them.

-------------------------------------------------------------------

* You know the extensive actually scientific stuff with proper controls**, statistical analysis, peer review and openness about methodology.

** For placebo effects***, age cohorts, other drugs, blah blah.

***Now if Homeopathy was to rebrand themselves as “Placebopathy” and drop the magic water-bashing of minute trace dilutions imparting some mystical quantum molecular memory mumbo jumbo then we’d be getting somewhere.

Thursday 3 February 2011

Questions for our Politicians

I just watched an utterly shameful display of political fuckwittery of the highest order. We had senior(ish) representatives from what are now apparently our four main political parties (FF, FG, SF & Labour) gracing our National airwaves in order to catastrophically fail to explain what their plans would be for tax increases and expenditure cuts if they happen to get into power. They seemed to prefer to call each other names and bang out shrill party slogans with obviously no intention of actually communicating anything substantive.

In the interests of getting this straight for myself (my numbers may be off by a billion here or there, it doesn’t make any difference) I took a look at the numbers.

We are currently running our country by spending €14bn more than we make. We make €33bn. In the long term a 3% (or ~€1bn) deficit is more or less OK so we’re searching for a way to tax more and spend less so that €13bn difference goes away by 2014 or thereabouts. Now this is down from a €17bn problem prior to December if I recall correctly. But that extra few billion will only remain saved if the incoming government does not backtrack significantly on the measures already taken by the outgoing FF crew.

So all of our friends on TV tonight needed to say where they will find that €14bn and none of them could offer up anything that convincingly demonstrated more than a few hundred million worth of ideas. At best they were talking about stuff they might do this year, not how they would actually get us out of this.

To put that number in some perspective, these are the important taxes and expenditures for 2007, 2010 and 2011. Between them they account for about 75% of the whole equation.

 

2010

2011

2007 (!!)

Income Tax

11.1

11.5

14.3

Corporation Tax

3.8

4.1

6.7

VAT

10.2

10.7

14.6

Excise\Stamp Duties

5.5

5.9

9.9

Education

-7.9

-8.4

-7.6

Social Welfare

-13.2

-14.2

-7.4

Health Services

-10.8

-10.6

-10.6

 

The 2007 numbers show just how extremely trashed the economy has been by the collapse – at that stage the four main sources of income exceeded the three main expenditure items by a more than 80%. Now income from them is 20% lower than the expenditure on those three alone.

Now ask yourself how any tweaking of the system can find €14bn in there without causing levels of hardship that are sure to cause a revolution.

Nobody is addressing this elephant in the room with any honesty – the numbers can be made add up but it’s got to be a combination of savage cuts in areas that haven’t seen real cuts yet (Education\Health), savage cuts in social welfare along with massive increases in Income tax, VAT and Excise. Corporation tax is a bust much as I would prefer it to be otherwise, any significant increase there will simply lead to offshoring of profits or disinvestment.

The problem with this is that the Irish population simply will not vote for someone who tells them that even if it is true and even if someone actually does manage to go so far as to implement it there will be blood on the streets. Note how outraged people are with the impact of the 2011 Budget, and just look at how trivial those changes actually were in the context of the immediate problem.

The other elephant in the room is that while growth could help make these numbers a lot more attainable the debt and interest burden required to fund ourselves until we get this into balance adds significantly to the problem – we’re running on a treadmill that’s accelerating there. And don’t forget that each €2bn in savings in expenditure translates into at least a 1% reduction in GDP if not more. So that’s another treadmill on top of the debt one that we have to race against.

Until someone puts some realistic numbers on genuine alternatives (wealth taxes that take into account the inevitable flight of wealth as a result, real property taxes, levies on natural resources (those gas fields off Mayo spring to mind) I can’t see how there’s any way out of this.

And I sure as hell don’t believe that any of our politicians have a clue how to do it either.

Tuesday 1 February 2011

Then and Now – Running a Country into the Ground.

Any monkey can run a business, or a country, when revenues are rising rapidly. It requires no skill to appear to be successful, because you can hide your mistakes easily and nobody asks hard questions when the money appears to be flowing. What does require skill and wisdom is recognising the danger that presents and acting pre-emptively to ensure that when your luck changes you are in a position to weather the storm.  Clearly none of our politicians have that skill, the promises made in advance of the last election are worth examining now as a measure of how they are all guilty.

I’m amazed that it now appears to be blazingly obvious to everyone that the tax base we had at the time was fragile and over exposed to construction. The fact that not one of the above documents gave the slightest indication that building a sustainable, recession proof tax base was on anyone’s mind a mere 9 months before the current disaster started to unwind is a good indication that the same people are not really equipped to get us out of this mess, regardless of party. Whatever options they put forward for consideration this time, they better show some better analysis of both sustainability and actual vision. It gets even worse if you go back further.

And don’t get me started on the Greens. Or Sinn Fein for that matter. At least the bloody PD’s did us all a favour and died out.

Primetime had a few notable numbers that we should all keep in mind with the election looming. Obviously we’re all aware of these, and the even more tangible reduction in net income after the last few budgets but it’s worth remembering that this is how badly screwed we are. And we would still be here no matter who had won the last election…

Despite last Decembers budget we are still running at expenditure levels that are almost 50% higher than revenues and this is before any of the punitive effects of IMF\EU “bailout” funding kicks in.

  2007 2011
Tax Revenues €48bn €33bn
Unemployment 158000 349000
National Debt €38bn €93bn
Deficit -€1bn (Surplus) €14bn (deficit)
Average House Price €340000 €191000

 

What I want to see is grovelling apologies across the board from all of these clowns and something that will genuinely convince me that they know how to [a] get us out of this and [b] make sure it can never happen again. 

Friday 7 January 2011

VHI and the drive to Co-Payment

The country has reacted to the VHI’s announcement of increases of up to 45% with a mixture of outrage and shock. Call centres from all three Health Insurers have buckled under the response as people scramble to find a way to either reduce or avoid this hit.

The thing that I find absolutely unbelievable in all of this is the continuous harping on about the burden that VHI’s elderly members put on the system. The amount of complete garbage being thrown about beggars belief. The VHI is apparently suffering because some 20% of their members are 60 or over, and those people tend to claim a lot more. This isn’t a surprise folks, that’s sort of why they have actuaries and all the other expensive risk profiling staff they have employed but even without them it’s blatantly obvious that older members will claim more. For a medical insurance firm to turn around and say that those people are a burden on them that affects their profits doesn’t make any sense – these people have been members for decades, probably all their lives, and were under claiming or claiming nothing for many of those years. If the VHI were properly accounting for the risks their members presented they should have built up reserves to deal with the age profile of their members. I’d have some sympathy for Quinn or Aviva making the claim given the fact that they haven’t been around so long but even so their charges should allow them to meet their expected claims profile, including making assumptions about those members getting older and making more claims.

If the VHI are actually claiming that the claims that older people now represent a 45% higher burden than they previously expected them to be then frankly they are admitting to gross incompetence. I’ve no problem with increases being necessary, medical costs are rising too fast, but the VHI and other providers are in a distinctly powerful position to keep that under control. In any case I can’t believe that’s true at all, it’s just a convenient excuse. The real problem is that Plan B represents a style of plan that the VHI really wants to kill off because such comprehensive insurance is a bad idea because it removes any cost awareness from the members, and ideologically that’s a bad thing if you’re an insurer.

It seems to me that the motivation is that the VHI want to move the population en masse to co-payment based plans, because those are believed to encourage lower overall claims, and to do so in a way that avoids most of the specific debate that would take place about that if they were to do it any other way. Given that the people who have reacted to this are, by and large, younger people on Plan B and its ilk I suspect that it is all pretty much going according to plan and the VHI will be pretty happy provided most people elect to change plans rather than providers. Even if relatively few elderly members switch they will have made significant inroads into eliminating the real problem – the 500,000 odd younger people that they never want to see growing old on a non co-payment plan.

If I’m right then VHI will continue making the same claims about the elderly being a burden , and continue to equate elderly with Plan B participation while they repeat the dramatic hikes on the cost of those plans.