Taha Ashtiani bio photo

Taha Ashtiani

Searching for EV in the tails.

City of toronto maintains a variety of interesting datasets on open.toronto.ca/datasets . In this article I explore the Active Permits and Cleared Permits datasets which contain records of all building permit applications of Toronto since 2000.

let’s start by loading the active permits dataset and taking a look at the features. I recommend you take a look at the readme file as well.

permits = pd.read_csv("datasets/activepermits.csv")


Most frequent types of permits



Most frequent street names


YONGE       5433
SHEPPARD    3819
BLOOR       3310
DUNDAS      3092
KING        3074
QUEEN       2965
LAWRENCE    2352
BAY         2160
EGLINTON    2140
ST CLAIR    1884
Name: STREET_NAME, dtype: int64

Contributions of occupancy

the last columns in the table are the square meters of the area for each type of occupancy. for example one of the permits has 157.00 square meters of RESIDENTIAL occupancy.

here are the percentages for the whole table.

occupancy = permits.loc[:,"ASSEMBLY":].sum(axis=0)
occupancy = round((occupancy/ occupancy.sum()) *100,2)
occupancy.sort_values(ascending=False).astype(str) + "%"

RESIDENTIAL                       42.95%
INTERIOR_ALTERATIONS              18.01%
INDUSTRIAL                        15.39%
DEMOLITION                         5.48%
ASSEMBLY                           2.58%
MERCANTILE                         1.94%
INSTITUTIONAL                      1.47%
dtype: object

Distribution of construction cost

Here we scale the construction costs to $1/1000 and then limit it to under 2 million dollars. The mean turns out to be at $506k and there are some picks on rounded values like $10k, $20k, $30k.

cost = permits["EST_CONST_COST"].dropna()
cost = cost[ cost.str.isnumeric()].astype(float)
cost /= 1000 # scale it to thousands of dollars
cost = cost[(cost>0) & (cost<1000)]
# cost = cost[(cost>0.001) & (cost<50)] # under $50k
ax = sns.distplot(cost,bins=20)
# ax.set(xticklabels=["","0k","$10k","$20k","$30k","$40k","$50k"])

count     89657.000000
mean        506.999859
std        6607.660147
min           0.000000
25%           1.000000
50%          15.000000
75%         120.000000
max      595875.683000
Name: EST_CONST_COST, dtype: float64

Most frequent structure types and permit types

pd.set_option('display.max_rows', 1000)
structure_permit_groups = permits.groupby(["STRUCTURE_TYPE","PERMIT_TYPE"])\
                                 .agg({ "PERMIT_TYPE":"count"})
structure_permit_groups["cnt_structure_type"] = structure_permit_groups.groupby(["INDX_STRUCTURE_TYPE"])\
structure_permit_groups.columns = ["cnt_permit_type","cnt_structure_type"]
structure_permit_groups = structure_permit_groups.sort_values(["cnt_structure_type","cnt_permit_type"]


Trend of permits over time

our main goal for this article is to gain some perspective on how the number and proportions of permits are changing over time. we start by looking into aggregate number of all permits together. The seasonal pattern is pretty interesting but doensn’t tell us anything about whats happening in each category .

permits["APPLICATION_DATE"] = pd.to_datetime(permits["APPLICATION_DATE"],format="%Y%m%d000000")
dates = permits["APPLICATION_DATE"].value_counts()
dates = dates.sort_index()
fig, ax  = plt.subplots(1)
fig.suptitle("number of permits over time",fontsize=20)


Categories of permits and their trend over time

First we find the most active types of permits. and then we can go ahead and plot each of them and see how they’re changing together.

frequent_permit_types= permits["PERMIT_TYPE"].value_counts().iloc[:5].index.tolist()

['Small Residential Projects',
 'Building Additions/Alterations',
 'New Houses']
fig, axes  = plt.subplots(1,figsize=(20,10))

fig.suptitle("types of permits over time",fontsize=20)

for i,permit_type in enumerate(frequent_permit_types):
    type_trend = permits[permits["PERMIT_TYPE"]==permit_type]["APPLICATION_DATE"].value_counts()
    ax = type_trend.rolling(100).mean().plot(label=permit_type)

plt.legend( prop={'size': 24})


The seasonal trend of Small Residential Projects is pretty strong. which totally makes sense in the case of outdoor projects. let’s take a look at the work column:

permits[permits["PERMIT_TYPE"]=="Small Residential Projects"]["WORK"].value_counts()

Multiple Projects                           18437
Interior Alterations                         7563
Addition(s)                                  6978
Garage                                       3504
Party Wall Admin Permits                     3299
Other(SR)                                    2702
Porch                                        2099
Deck                                         1916
Underpinning                                 1731
Unknown                                      1413
Accessory Building(s)                         820
Second Suite (New)                            715
Walk-Out Stair                                651
Fire Damage                                   532
Finishing Basements                           389
Carport                                       326
Canopy                                        281
New Laneway / Rear Yard Suite                 111
Pool Fence Enclosure                           98


the CURRENT_USE column gives us a good insight on what most of these permits are all about. let’s take a quick look at the most frequent ones.


Sfd                       48593
Vacant                    26441
Office                    14141
Sfd-Detached              10851
Single Family Dwelling     6513
Retail                     5993
Sfd - Detached             4917
Apartment Building         3835
Sfd Detached               3620
Sfd-Semi                   2767
Industrial                 2596
Apartment                  2186
Vacant Land                1997
Restaurant                 1997
Hospital                   1657
Sfd-Townhouse              1284
House                      1276
Single Family Detached     1236
Retail Store               1235
Sfd Semi                   1235
Name: CURRENT_USE, dtype: int64

Focusing on new houses and new buildings

let’s take a look at the zip codes and streets with the most permit applications. later on we’ll talk about how this can be important.

new_buildings = permits[permits["PERMIT_TYPE"].isin(["New Building","New Houses"])]
active_zips =  new_buildings.groupby(["POSTAL","STREET_NAME"]).agg({"APPLICATION_DATE":"count"})\


zips = permits["POSTAL"].unique()

array(['M2R', 'M4L', 'M6R', 'M6K', 'M6H', 'M5C', 'M4K', 'M6P', 'M2N',
       'M2P', 'M1S', 'M6B', 'M9C', 'M5B', 'M2J', '   ', 'M5J', 'M8V',
       'M5P', 'M3A', 'M5G', 'M5T', 'M4G', 'M3M', 'M9W', 'M8Z', 'M5M',
       'M4M', 'M4E', 'M6J', 'M6G', 'M4Y', 'M5V', 'M2L', 'M8Y', 'M4W',
       'M4N', 'M6L', 'M9A', 'M4V', 'M5N', 'M2M', 'M1L', 'M4T', 'M6S',
       'M6C', 'M5A', 'M4C', 'M9B', 'M3C', 'M5S', 'M4B', 'M9V', 'M1P',
       'M6A', 'M4A', 'M6E', 'M6N', 'M8X', 'M4P', 'M3B', 'M3H', 'M4R',
       'M6M', 'M5H', 'M3J', 'M9M', 'M8W', 'M1C', 'M2H', 'M9R', 'M9N',
       'M4J', 'M1E', 'M5R', 'M4X', 'M3N', 'M4S', 'M5E', 'M5X', 'M2K',
       'M9L', 'M1X', 'M1B', 'M3L', 'M3K', 'M9P', 'M1N', 'M1T', 'M1G',
       'M1K', 'M7A', 'M1J', 'M1H', 'M1R', 'M1M', 'M1W', 'M4H', 'M1V'],

Most active zip codes over time

here we start by finding and sorting the most active zip codes, based on the total number of applications. and then we get the top 10 and look into their trend over time. filtering out the most active zip codes gives us meaningful trends.

most_active_zips = new_buildings.groupby(["POSTAL"])\

most_active_zips = most_activezips.index[1:10]   
Daily_Count_zipcodes = new_buildings.groupby([pd.Grouper(key="APPLICATION_DATE",freq="M"),"POSTAL"])\

Daily_Count_zipcodes.columns = ["POSTAL_COUNT"]
# Daily_Count_zipcodes
fig, ax = plt.subplots(1,figsize=(20,15))

for zip_code in most_active_zips:

plt.legend( prop={'size': 24})