December

19

ColdFusion and Google Analytics: Getting Out What You Put In

The Hooking into Google Analytics post details how to connect to the Data Export API of Google Analytics (GA) and get a simple visitors count in return. Here we will expand greatly on that and create a cfc that will process just about any result from the API.

This code requires at least ColdFusion 8.0.1. Check your version before installing.

An auth token is needed for requests to the API and two ways to do that are the AuthSub and ClentLogin methods. The method used is up to the developer as once the auth token is retrieved and set into session, data calls to the API can be repeatedly made.

<cffunction name="googleLogin" access="public" hint="GA account authorization">
        <cfargument name="email" type="string" required="yes" default="">
        <cfargument name="password" type="string"required="yes" default="">
        <cfargument name="gaLoginUrl" type="string" required="no" default="https://www.google.com/accounts/ClientLogin">

        <cfset var loginAuth = "" />

        <cfhttp url="#arguments.gaLoginUrl#" method="post">
            <cfhttpparam name="accountType" type="url" value="GOOGLE">
            <cfhttpparam name="Email" type="url" value="#arguments.email#">
            <cfhttpparam name="Passwd" type="url" value="#arguments.password#">
            <cfhttpparam name="service" type="url" value="analytics">
            <cfhttpparam name="source" type="url" value="my-analytics">
        </cfhttp>

        <cfif NOT FindNoCase("Auth=",cfhttp.filecontent)>
            <cfset loginAuth = "Authorization Failed" />
        <cfelse>
            <cfset loginAuth = Mid(cfhttp.filecontent, FindNoCase("Auth=",cfhttp.filecontent) + (Len("Auth=")), Len(cfhttp.filecontent)) />
        </cfif>

         <cflock scope="session" type="exclusive" timeout="5">
                <cfset session.ga_loginAuth = loginAuth />
         </cflock>

    </cffunction>

To retrieve the profile or data feed, a call is made to the API via a function.

   <cffunction name="callApi" access="public" returntype="array" hint="GA data as array of structures">
        <cfargument name="gaUrl" type="string" required="yes">
        <cfargument name="authToken" type="string" required="no" default="#session.ga_loginAuth#" />

        <cfset var authTokenHeader = 'GoogleLogin auth=' & arguments.authToken />

        <cfset var responseOutput = "" />

        <cfhttp url="#arguments.gaUrl#" method="get">
            <cfhttpparam name="Authorization" type="header" value="#authTokenHeader#">
        </cfhttp>

        <cfset responseOutput = cfhttp.filecontent />
        <!---remove dxp: prefix from nodes that have it and strip xmlns from feed element--->
         <cfset responseOutput = responseOutput.ReplaceAll("(</?)(\w+:)","$1") />
         <cfset responseOutput = REReplaceNoCase(responseOutput,"<feed[^>]*>","<feed>") />
         <!---entry nodes hold the data--->
         <cfset entryNodes = XmlSearch(responseOutput, '//entry/') />

         <cfreturn entryNodes />
    </cffunction>

Next, a list of the profiles (websites) that the user has access to are retrieved by requesting an account feed. If the user has only one profile associated with their credentials, the data is displayed immediately. If they have more than one, they are presented with a select box to select the profile they want data from.

<cfinvoke component="ga" method="parseProfiles" returnvariable="profilesArray"></cfinvoke>
 <cffunction name="parseProfiles" access="public" returntype="array" hint="GA profiles as array of structures">

	<cfset var profileArray = ArrayNew(1) />
        <cfset var entryStruct = StructNew() />

        <cfset entryNodes = callApi("https://www.google.com/analytics/feeds/accounts/default") />

        <cfloop array="#entryNodes#" index="entry">
            <cfset entryStruct = StructNew() />

            <cfset entryStruct.title = entry.title.XmlText />
            <cfset entryStruct.tableId = entry.tableId.XmlText />

            <cfset arrayAppend(profileArray,duplicate(entryStruct)) />
        </cfloop>

        <cfreturn profileArray />
    </cffunction>

A default date range of one year is set initially. A dialog box that allows the user to change the date range is presented as an option. The date range is limited to a max date of yesterday because GA does not have full data for the current day. This causes the averages that are calculated on the return data to be skewed.

A data feed request url is sent to GA along with the auth token a via cfhttp tag.

<cfset reqUrl = "https://www.google.com/analytics/feeds/data?ids=" & session.tableId & "&metrics=ga:newVisits,ga:pageviews,ga:visits,ga:visitors,ga:timeOnSite&start-date=" & session.startdate & "&end-date=" & session.enddate />

The XML response is then parsed out and returned as an array of structures.

<cffunction name="parseData" access="public" hint="GA data as array of structures set in session">
        <cfargument name="gaUrl" type="string" required="yes" />
        <cfargument name="arrayName" type="string"required="yes" />

        <cfset var dataArray = ArrayNew(1) />
        <cfset var entryStruct = StructNew() />

         <cfset entryNodes = callApi(arguments.gaUrl) />

        <!---CF8 loop through the entries and put each data point in structure--->
        <cfloop from="1" to="#ArrayLen(entryNodes)#" index="num">
		<!---rest of the stats data from GA, first check if dimension exists--->
		<cfif StructKeyExists(entryNodes[num],"dimension")>
		<cfloop from="1" to="#ArrayLen(entryNodes[num].dimension)#" index="i">
		<!---start after ga: to remove it--->
		<cfset "entryStruct.#Mid(entryNodes[num].dimension[i].XmlAttributes["name"],4,
Len(entryNodes[num].dimension[i].XmlAttributes["name"]))#" = entryNodes[num].dimension[i].XmlAttributes["value"] />
		</cfloop>
		</cfif>

		<cfloop from="1" to="#ArrayLen(entryNodes[num].metric)#" index="i">
			<cfset "entryStruct.#Mid(entryNodes[num].metric[i].XmlAttributes["name"],4,
Len(entryNodes[num].metric[i].XmlAttributes["name"]))#" = entryNodes[num].metric[i].XmlAttributes["value"] />
		</cfloop>

		<cfset arrayAppend(dataArray,duplicate(entryStruct)) />

	</cfloop>

         <!---CF9 can use this simplier loop. If you have CF9, uncomment and delete loop above
            <cfloop array="#entryNodes#" index="entry">
		<cfif StructKeyExists(entry,"dimension")>
                	<cfloop array="#entry.dimension#" index="dimension">
                 	<cfset "entryStruct.#Mid(dimension.XmlAttributes["name"],4,
Len(dimension.XmlAttributes["name"]))#" = dimension.XmlAttributes["value"] />
                 	</cfloop>
                 </cfif>

                 <cfloop array="#entry.metric#" index="metric">
                 	<cfset "entryStruct.#Mid(metric.XmlAttributes["name"],4,
Len(metric.XmlAttributes["name"]))#" = metric.XmlAttributes["value"] />
                  </cfloop>

                <cfset arrayAppend(dataArray,duplicate(entryStruct)) />
           </cfloop>--->

        	<cflock scope="session" type="exclusive" timeout="5">
        		<cfset "session.#arrayName#" = dataArray />
        	</cflock>

    </cffunction>

Once the data is in a form you can manipulate, graphs, tables, and charts of all kinds can be made.

ClientLogin, AuthSub, and Secure AuthSub authentication examples are included in the demo. ClientLogin recommended only for client apps. Google notice on using ClientLogin:

“Important: Do not use ClientLogin if you are writing an application that runs on
your computer to make requests on behalf of 3rd party end users. Instead, use either
AuthSub or OAuth, which protects end users’ private data. Because ClientLogin stores
the user login data, it should only be used in cases where that data is under the
direct control of the user (e.g. their personal computer).”

Recommended Reading

Demo

Download files from github

 

Further Reading:

  1. Hooking into Google Analytics with ColdFusion
  2. Google Analytics API Login Authentication with ColdFusion
  3. Generating Signatures in ColdFusion with RSA-SHA1 for Secure AuthSub in Google Analytics

25 Comments for ColdFusion and Google Analytics: Getting Out What You Put In


Benedict Lowndes
February 19, 2010

Great work! Have you got any interest in packaging this up as a download, or even better, adding it to riaforge or github?


jen
February 20, 2010

@Benedict

Working on it.

Thanks!


Francesco
May 5, 2010

Where is the code link download?

I don't understand where to user the reqUrl variable… and how to make the logout button!


jen
May 15, 2010

@Benedict @Fransesco,

Now on github via the Download link in the post.


Benedict Lowndes
May 15, 2010

Excellent, thanks Jen!


ruben
June 22, 2010

Hi, when i run the application i have the next error after select the Google Analytics profile.
Element STATSDATAARRAY is undefined in a Java object of type class [Ljava.lang.String;.
Can you help me?
Thanks


jen
June 22, 2010

@ruben
If cfthread is in the code, remove all references to it and run it again. I need to update the code so cfthread runs consistently.


jen
June 22, 2010

@ruben
I updated the code on github. I just removed cfthread for now.


James
June 23, 2010

@Jen,

This is a wonderful example, and I love how it works.
Just a quick question, does it work on Codlfusion8? My 'live' server is running coldfusion 8 and it throws an error, however on my local box (on CF9) it works without incident.


jen
June 23, 2010

@James
Check your version of CF8. It needs at least 8.0.1.


James
June 23, 2010

Ok it's currently 8,0,1,195765 I probably did something wrong on my server config.


jen
June 23, 2010

@James
Send me the error if you can't figure it out.


James
June 23, 2010

The error is as follows:

The web site you are accessing has experienced an unexpected error.
Please contact the website administrator.

The following information is meant for the website developer for debugging purposes.
Error Occurred While Processing Request
Element STATSDATAARRAY is undefined in a Java object of type class [Ljava.lang.String;.


jen
June 23, 2010

@James
Couple of things to try. Make sure cfthread is not in the code on the index.cfm page. And, make sure your session vars are cleared. Run login.cfm?logout=true in the browser to clear the session vars and start fresh.


Ruben
June 23, 2010

Hi Jen, with the new index.cfm the StatsDataArray problem was solve, but now i have the next error:
Error Occurred While Processing Request
Element XMLATTRIBUTES is undefined in a Java object of type class [Ljava.lang.String;.

Thanks


James
June 23, 2010

@Jen,
Tried both of those things, and I still get the same error. Any additional Thoughts???


James
June 23, 2010

@Jen,
I meant the same error as Ruben is recieveing.


jen
June 23, 2010

@James @Ruben
Try this in the ga.cfc parseData function. You will have to replace the loop that is there:

<cfloop from="1" to="#ArrayLen(entryNodes)#" index="num">

<cfif StructKeyExists(entryNodes[num],"dimension")>
<cfloop from="1" to="#ArrayLen(entryNodes[num].dimension)#" index="i">
<cfset "entryStruct.#Mid(entryNodes[num].dimension[i].XmlAttributes["name"],4,
Len(entryNodes[num].dimension[i].XmlAttributes["name"]))#" = entryNodes[num].dimension[i].XmlAttributes["value"] />
</cfloop>
</cfif>

<cfloop from="1" to="#ArrayLen(entryNodes[num].metric)#" index="i">
<cfset "entryStruct.#Mid(entryNodes[num].metric[i].XmlAttributes["name"],4,
Len(entryNodes[num].metric[i].XmlAttributes["name"]))#" = entryNodes[num].metric[i].XmlAttributes["value"] />
</cfloop>

<cfset arrayAppend(dataArray,duplicate(entryStruct)) />

</cfloop>


James
June 23, 2010

@Jen:

Thanks for that new Loop. It works now. Very Odd that the old one didn't but this one does. Very Awesome, thanks Very Much Jen!


jen
June 23, 2010

@James
It works fine on CF9 as is, but I will update the git repository tonight with the loop I posted above.


jen
June 23, 2010

Git repository updated with both versions of loop so you can choose which to go with.


Ruben
June 23, 2010

Jen, your new code works excelent, thanks a lot for your help, is a very good example to understand the connections between Google analytics and coldfusion. Do you know were can i find the other Google Analytics variables to make some tests? for example Goals Variables? Thanks a lot again.


jen
June 24, 2010

@Ruben
Glad to hear the code is working out for you. Here are the resources I use the most for getting the data out of GA:
Dimension and Metrics List: http://code.google.com/apis/analytics/docs/gdata/gdataReferenceDimensionsMetrics.html
Data Query Feed Explorer (most helpful): http://code.google.com/apis/analytics/docs/gdata/gdataExplorer.html
The Data Query Feed Explorer will actually give you the query url you need to use for the information you want returned.
Good luck.


Ruben
June 24, 2010

@Jen Thanks a lot for everything, do you work as a freelance in case that i have a coldfusion project?


jen
June 24, 2010

@Ruben
Yes, I do. Let me know if you need anything.

Leave a comment

Why ask?

 

« | »