What’s the Difference Between GDA94 and GDA2020?

What’s the Difference Between GDA94 and GDA2020?

Feb 11, 2025 | Data Domain

Geodetic datums, or geodetic systems, are often used by proptechs for mapping  or analysing spatial data.

Here is a rundown of everything you need to know about the different geodetic datums we use and reference in Australia.

Table of Contents

Jump ahead ↓

What is a Geodetic Datum?

A geodetic datum is a reference framework used to define the Earth’s shape and orientation, providing a coordinate system that allows for accurate mapping, surveying, and pinpointing exact locations on the Earth’s surface.

In Australia, we use Geodetic Datum of Australia 1994 (DA94) and Geodetic Datum of Australia 2020 (GDA2020).

The History of Australia’s Geodetic Datums

Prior to GDA94, Australian surveyors primarily used the Australian Geodetic Datum 1966 (AGD66), which was based on a network of ground-based survey points and astronomical observations.

AGD66 was the standard datum used for mapping and surveying in Australia for several decades until it was superseded by GDA94 in the 1990s.

The decision to switch to GDA94 was driven by the need for a more accurate and up-to-date geodetic datum that could take advantage of advances in geospatial technology such as GPS. AGD66 was also affected by tectonic movements and other changes in the Earth’s surface, which made it increasingly difficult to use for accurate positioning and navigation.

GDA94 (Geocentric Datum of Australia 1994) was the geodetic datum used in Australia from 1994. Based on a mathematical model of the Earth’s surface defined using measurements from a network of ground-based survey points, and used as the standard datum for mapping and surveying in Australia.

Now, GDA2020 (Geocentric Datum of Australia 2020) is the current geodetic datum used in Australia. It was introduced in 2017 to replace GDA94 and is based on more recent measurements of the Earth’s surface using advanced satellite and ground-based technology.

GDA2020 provides a more accurate representation of the Earth’s surface than GDA94, and is designed to be compatible with global positioning systems (GPS) and other modern geospatial technologies.

Even though AGD66, and to some extent GDA94, are no longer the primary datums used in Australia, it’s still important to maintain historical data that was referenced to this datum. And it is possible to transform data from AGD66 to GDA94 or GDA2020 using appropriate transformation parameters to ensure compatibility and accuracy when comparing or integrating data from different sources.

Conversions between Geodetic Datums

Conversions between AGD66 and GDA94 are not 100% accurate, because the two datums are based on different mathematical models of the Earth’s surface with different reference points and parameters.

To convert data from AGD66 to GDA94 (or vice versa), a mathematical transformation must be applied that takes the differences between the two datums into account.

This transformation involves adjusting the latitude, longitude and height values of the data to align with the new datum.

However, there are many factors that can affect the accuracy of this transformation, such as:

  1. The quality and accuracy of the original data: If the original data was collected using imprecise or inaccurate methods, the transformation may introduce additional errors or inaccuracies.
  2. The complexity of the transformation: Some transformations may require more complex mathematical models or additional parameters to be specified, which can increase the likelihood of errors.
  3. The location and terrain of the data: The accuracy of the transformation can vary depending on the location and terrain of the data. Some areas may be more affected by tectonic movements or other changes in the Earth’s surface, which can make the transformation more challenging.
  4. The type of data being transformed: Different types of data (e.g. points, lines, polygons) may require different transformation methods or parameters, which can affect the accuracy of the transformation.

While conversions between AGD66 and GDA94 can be relatively precise, they’re not 100% accurate.

This is due to the inherent differences between the two datums, and the potential for errors or inaccuracies in the transformation process. It’s important to use appropriate transformation methods and understand the limitations and potential sources of error when converting data between different geodetic datums.

The Difference Between GDA94 and GDA2020

The key differences

The main difference between GDA94 and GDA2020 is their accuracy and the methods used to define them.

GDA2020 is a more accurate and up-to-date datum, with improvements in the modeling of the Earth’s surface that take into account changes in its shape over time. This means that positions and distances measured using GDA2020 are more accurate than those measured using GDA94. Additionally, GDA2020 is designed to be compatible with modern geospatial technologies and is expected to be used for many years to come.

It’s worth noting that the difference between GDA94 and GDA2020 may not be significant for many applications, particularly those that don’t require high levels of accuracy. However, for applications that require precise positioning or measurement, such as surveying or mapping, selecting the correct geodetic datum is important to ensure accurate results.

Differences in distance and direction

The average distance and direction difference between GDA94 and GDA2020 depends on the location on the Earth’s surface.

In general, the differences between the two datums are greatest in areas with high tectonic activity or areas where the Earth’s surface is undergoing significant changes, such as due to land subsidence or sea level rise.

According to Geoscience Australia, the organisation responsible for geodetic information and services in Australia, the average difference between GDA94 and GDA2020 in Australia is around 1.5 meters. However, this value can vary significantly depending on the location, with some areas showing differences of several meters or more.

The direction of the difference between the two datums also varies depending on the location, as it is related to the direction and magnitude of any tectonic movements or changes in the Earth’s surface. In general, the direction of the difference is determined by the vector between the two datums at a given location.

It’s important to note that the difference between GDA94 and GDA2020 is not constant over time and may continue to change in the future. This is because the Earth’s surface is constantly changing due to tectonic activity, sea level rise, and other factors. As such, it’s important to regularly update geodetic data and use the most up-to-date geodetic datum for accurate positioning and navigation.

Migrating from GDA94 to GDA2020

The differences between the two means that migrating from GDA94 to GDA2020 can present several challenges and issues, particularly for organisations or projects that rely heavily on geospatial data.

Some of the key issues with migrating to GDA2020 include: 

  1. Data compatibility: Data that was created using GDA94 may not be compatible with GDA2020. This can cause issues when trying to integrate or compare datasets that use different datums.
  2. Application compatibility: Applications that were designed to work with GDA94 may not be compatible with GDA2020. This can require updates or modifications to existing software or the adoption of new tools.
  3. Training and expertise: Staff who work with geospatial data may need to be trained on the new GDA2020 datum and its associated tools and workflows. This can take time and resources.
  4. Time and cost: Migrating to GDA2020 can be a complex and time-consuming process, particularly for large organisations or projects. There may be costs associated with updating software, purchasing new equipment, or retraining staff.
  5. Accuracy: While GDA2020 is a more accurate datum than GDA94, some existing data may still be more accurate when referenced to GDA94. This can make it difficult to compare or integrate data from different sources.
  6. Data transformation: In some cases, it may be necessary to transform data from GDA94 to GDA2020, which can introduce errors or inaccuracies. The accuracy of the transformation depends on the quality of the original data and the transformation method used.

Migrating from GDA94 to GDA2020 requires careful planning and consideration of the potential issues and challenges. It’s crucial to work closely with geospatial experts and stakeholders to ensure a smooth and successful transition.

What is WGS84 and Why is it Used by Software?

WGS84 (World Geodetic System 1984) is a geodetic datum used for positioning and navigation purposes. It defines a reference system for the Earth’s surface that allows locations to be specified in latitude and longitude coordinates.

The WGS84 datum was developed by the United States Department of Defense for use by the military and intelligence agencies, but it has since become the standard geodetic datum used by many organisations and applications around the world, including GPS (Global Positioning System) devices and mapping software.

The WGS84 datum is based on a mathematical model of the Earth’s surface that takes into account its shape, size, and rotation. It defines a set of reference points and parameters that allow positions on the Earth’s surface to be accurately calculated and communicated.

The WGS84 datum is widely used because it is compatible with GPS and other global navigation systems, allowing precise positioning and navigation in real-time. However, while there may be regional differences in the Earth’s surface that are not fully captured by the WGS84 model, that other geodetic datums may be more appropriate for certain applications or regions.

How to Convert Between GDA2020 and WGS84

To convert between GDA2020 (Geocentric Datum of Australia 2020) and WGS84 (World Geodetic System 1984), you can use coordinate transformation parameters provided by geodetic authorities. The transformation process involves converting coordinates from one datum to another using a mathematical model.

In the case of GDA2020 and WGS84, the transformation parameters provided by the Intergovernmental Committee on Surveying and Mapping (ICSM) in Australia are known as the National Transformation Version 2 (NTv2) grid files. These grid files contain the necessary information for accurate transformations.

The accuracy of the transformation depends on the specific region and the quality of the NTv2 grid files used. Always use the most up-to-date and accurate transformation parameters available from reputable sources.

To convert coordinates between the GDA2020 (Geocentric Datum of Australia 2020) and WGS84 (World Geodetic System 1984) datums using Python, you can utilise the pyproj library. pyproj provides a convenient interface to the PROJ library, which is a widely used cartographic projection and coordinate transformation library.

Geodetic Datum Usage in Australia

In Australia, a lot of data providers offer data sets in both GDA94 and GDA2020 geodetic datums because the uptake of GDA2020 is not universal. It’s common practice for these providers to specify which datum was used to create each dataset.

When combining geospatial datasets, it’s important for data professionals to ensure consistency in the geodetic datums employed.

Using different datums without proper alignment can lead to inaccuracies, such as misaligning spatial features. For this reason, careful attention to datum consistency is essential to maintain the integrity and accuracy of integrated geospatial data.

 

Originally published: 5 August, 2023

Last updated: 11 February, 2025

 

Read more from The Proptech Cloud

Understanding Housing Affordability: Key Metrics and Statistics

Housing affordability is a significant concern in many parts of the world, affecting the quality of life and economic wellbeing of individuals and families.

To understand the dynamics of housing affordability we need to take a detailed look at a range of different metrics and statistics to gain a full picture.

Read more

The Three Primary Methods of Real Estate Data Integration

cLearn the three primary methods of real estate data integration—geospatial relationships, title matches, and address matching—to improve accuracy, insights, and decision-making.

Read more

What’s the Difference Between GDA94 and GDA2020?

Geodetic datums, or geodetic systems, are often used by proptechs. Here is a rundown of everything you need to know about the different geodetic datums we use and reference in Australia.

Read more

Alternative Data: What Is It, Who Uses It And Why It Matters

Discover the powerful intel alternative data can offer. Learn why businesses and investors are turning to non-traditional data sources for deeper insights and smarter decisions.

Read more

What Is A Geohash And How Is It Used?

Discover what a geohash is, how it works, and its real-world applications in mapping, logistics and data analysis.

Read more

What Is A Cadastre?

What Is A Cadastre?

Nov 18, 2024 | Data Domain

Cadastres are used extensively in real estate and beyond. We break down what they are, how they’re stored, used and maintained in Australia.

Cadastre At A Glance (TL;DR)

A cadastre is an official record of land ownership, boundaries and value, crucial for property management, taxation and legal clarity.

Traditional cadastral systems relied on paper maps and manual record-keeping, whereas modern cadastres keep property data accurate, interactive and accessible.

Cadastres are often created and managed with technologies like Geographic Information System (GIS) which capture, store, analyse and visualise spatial or geographic data.

Around the world, innovations like 3D cadastres and digital platforms are enhancing how we manage, visualise and understand land data.

Table of Contents

Jump ahead ↓

What Is A Cadastre?

A cadastre is a detailed register or database that holds information about land or property within a specific area. Each record in the cadastre defines the boundaries of a property, as mapped by land surveys. It also includes important details like the property’s location, features, and value.

A cadastre is an essential tool for managing and regulating land ownership and use. It helps with collecting property taxes, assessing land values, and resolving property disputes. It usually includes property boundaries, ownership details, and physical descriptions like size, shape, and terrain. It may also contain information on land use, zoning rules, building permits and environmental regulations.

What Does A Cadastre Look Like On A Map?

The way a cadastre is represented on a map can vary significantly depending on the source of the data and the configuration settings used in the mapping software. Different factors like styling, symbols, and labeling options can influence how the cadastre appears visually on the map.

The following image is a typical representation of cadastre on a map, showing boundary lines that delineate the various land parcels or lots. These boundary lines help visually separate one property from another. While lot numbers are used as an identifying label to provide a quick reference to specific parcels within the cadastre.

Cadastre represented on a map

Australian Cadastres

In Australia, individual state and territory governments are responsible for the maintenance of cadastres, rather than the federal government.

Each state and territory has its own land administration agency responsible for maintaining cadastres within their jurisdiction.

The following organisations maintain cadastres:

These agencies are responsible for updating and managing the cadastre, including recording changes to property ownership, boundaries, and other relevant information. They provide access to the cadastre and offer related services to the public, including title searches, property reports, and other land-related information.

What File Types Are Used To Store Cadastres?

These file types are commonly used to store a cadastre:

  • Shape file (.shp)
  • GDB (.gdb)
  • Geojson (.geojson)

File sizes of cadastre files can become quite large, depending on the extent of the coverage.

For example, the cadastre for New South Wales (NSW) in Australia is approximately 1.4 GB when compressed. Working with large files can be more manageable in cloud environments like Amazon Web Services (AWS). These cloud platforms provide substantial computing power that can be accessed when needed, then switched off in a pay-per-use model to more efficiently handle the processing requirements of large cadastre files.

Primarily spatial files, they contain geometry data that represents the boundaries of each land parcel within the cadastre. The geometry information can be stored and represented in various text formats, which are universally understood by spatial data software applications.

The most common approaches for storing and representing the geometries are

which ensure compatibility and ease of interpretation across different software tools and platforms.

Additional attributes relating to the cadastre can also be served within the same spatial file, such as through the properties key within a cadastre’s GeoJSON document. Other formats such as WKT or WKB do not support the direct inclusion of additional attributes to the geometry, but can be associated with in different ways such as in an accompanying csv file containing any additional attributes.

What Does A Cadastre Look In A Snowflake Marketplace Listing?

To incorporate a cadastre into Snowflake, it needs to be transformed into a table structure. The process involves loading the cadastre data in the form of GeoJson as a VARIANT data type in Snowflake. Then the GeoJson features are flattened and converted into individual rows within the table.

Alternatively, the cadastre file can be converted to a flat file outside of Snowflake, then loaded into Snowflake as you would with any other flat file.

This flattening process makes it easier to query and analyse the cadastre data using standard SQL operations within Snowflake, allowing for efficient storage, retrieval, and analysis of the information.

Attribute {A}Attribute {B}Attribute {C}Geometry
123POLYGON((30 10, 40 40, 20 40, 10 20, 30 10))

*The actual columns (feature attributes) available for each piece of land registered on a cadastre is dependent on the maintainer/publisher of the cadastre.

Who Provides Cadastres On The Snowflake Marketplace?

Here are a few providers of cadastres on the Snowflake Marketplace:

  • The Proptech Cloud
  • Geoscape
  • Precisely

How Are Cadastres Used?

Cadastres play a pivotal role in linking spatial data to real-world applications.

In the context of the built environment, cadastres serve a number of purposes, such as:

  • Identify the unique number of properties in a country,

  • Identify changes to properties (merges, subdivisions, title registrations),

  • Spatially link other spatial information to a property.

  • Spatially lookup a property based, i.e. lookup properties based on latitude and longitude coordinates, or based on geospatial shape (think drawing a circle on map to search for properties on the map).

  • Represent property boundaries on a map.

  • Assess property risks, develop climate change adaption strategies, evacuation routes and emergency responses.

  • Effective land use planning to guide urban development and expansion, the management of rural land resources and supporting environmental protection initiatives.

  • Planning and managing infrastructure projects, including tility networks.

The Future of Cadastres

Around the world, countries are adopting 3D cadastres to better capture the complexity of modern property landscapes.

For example, a collaborative project between Russia and the Netherlands explores how 3D models can improve the recording of rights associated with multi-level buildings, complex structures, and underground networks, such as gas pipelines.

The Netherlands has also developed 3D cadastre solutions to address the limitations of 2D cadastral maps in representing complex spatial property arrangements.

In Australia, the Australian CADASTRE 2034 strategy aims to create a fully digital, interoperable cadastre that can support various applications beyond traditional land administration.

As technology advances, so too does our ability to understand and shape the spaces in which we live and grow.

These advancements are an inspiring shift toward a future where cadastres will play a bigger role in urban planning, environmental management, and beyond.

Originally published: 25 September, 2023

Last updated: 19 November, 2024

Subscribe to our newsletter

Subscribe to receive the latest blogs and data listings direct to your inbox.

Read more from The Proptech Cloud

Understanding Housing Affordability: Key Metrics and Statistics

Housing affordability is a significant concern in many parts of the world, affecting the quality of life and economic wellbeing of individuals and families.

To understand the dynamics of housing affordability we need to take a detailed look at a range of different metrics and statistics to gain a full picture.

Read more

The Three Primary Methods of Real Estate Data Integration

cLearn the three primary methods of real estate data integration—geospatial relationships, title matches, and address matching—to improve accuracy, insights, and decision-making.

Read more

What’s the Difference Between GDA94 and GDA2020?

Geodetic datums, or geodetic systems, are often used by proptechs. Here is a rundown of everything you need to know about the different geodetic datums we use and reference in Australia.

Read more

Alternative Data: What Is It, Who Uses It And Why It Matters

Discover the powerful intel alternative data can offer. Learn why businesses and investors are turning to non-traditional data sources for deeper insights and smarter decisions.

Read more

What Is A Geohash And How Is It Used?

Discover what a geohash is, how it works, and its real-world applications in mapping, logistics and data analysis.

Read more

Crafting a Storm Surge and Hurricane Risk Rating for Coastal Properties

Crafting a Storm Surge and Hurricane Risk Rating for Coastal Properties

Jul 23, 2024 | Inference and Analytics

In an era where climate change is intensifying the frequency and severity of storms and hurricanes, especially in coastal regions, understanding and quantifying the associated risks is critical.

According to the National Geographic Society, a storm surge is a rise in sea level that occurs during tropical cyclones, which are intense storms also known as typhoons or hurricanes.

The storms produce strong winds that push the water into shore which can lead to flooding and pose a real threat in coastal regions.

To help understand these risks, a Storm Surge and Hurricane Risk Rating score can provide property owners, developers, real estate agents, insurers, urban planners, local governments, buyers and investors with a clear picture of a property’s vulnerability to these natural disasters.

These stakeholders will be conducting their own necessary research, and a risk rating system can offer an indicative metric to guide their decisions.

Why is a Storm Surge and Hurricane Risk Rating Important?

Understanding storm surge and hurricane risks is crucial for building a resilient society.

Natural catastrophes pose significant challenges, and quantifying these risks can aid in better preparation and prompt responses.

Strengthening homes and incentivising homeowners to invest in property fortification can reduce potential losses. Accurate risk assessments and reliable data can allow insurers to offer discounts for mitigation actions, enhance home resale values, and reveal the increased costs to mortgage issuers due to natural disasters.

Achieving resilience relies on expert understanding of the real estate ecosystem and the benefits of informed mitigation strategies.

Steps to build a Storm Surge and Hurricane Risk Rating

1. Defining scoring criteria and scale

The foundation of a risk rating system is a clear and understandable scale, such as 1 to 10, with each number representing a different level of risk.

Establishing specific criteria for assessment is also essential for a well-rounded evaluation.

2. Key factors to consider

Several factors contribute significantly to a property’s risk from storm surges and hurricanes:

Proximity to coast

 

  • Proximity to Coastline: The closer a property is to the coastline, the higher the risk of storm surge impacts.
  • Elevation and Topography: Properties at higher elevations or with certain topographical features may have reduced risk.
  • Historical Data: Analysing past hurricane and storm surge incidents from historical weather databases and local government records can provide critical insights into potential future risks.
  • Local Climate Trends: Understanding the local weather patterns can help predict the likelihood of storms.
Natural barriers
  • Flood Zone Designation: Properties in designated flood zones face a heightened risk. Flood risk information is generally available from Local Councils.
  • Building Design and Materials: Construction that is designed to be resilient against high winds and flooding can mitigate risk.
  • Infrastructure and Preparedness: Robust local infrastructure and emergency plans can play a vital role in risk reduction.
  • Natural Barriers: The presence of natural features, such as dunes or wetlands that can absorb storm impacts, reduces risk.
  • Regional Planning: Effective community and regional planning and zoning can mitigate potential damage. Consult local zoning laws and development plans for more property-specific.

3. Assigning weights to each factor

Assigning appropriate weights to each of the above factors based on its impact on overall risk ensures that the score accurately reflects the property’s vulnerability.

Use expert consultations and statistical analysis to determine appropriate weights, and adjust weights based on real-world data and expert feedback.

4. Data collection and analysis

Gathering and analysing data, including GIS mapping, climate records and historical event data, is crucial to assigning accurate sub-scores for each criterion. Cross-referencing multiple sources will ensure data accuracy and statistical software can be used for thorough analysis.

5. Calculating the overall score

By aggregating these sub-scores, considering their respective weights, we arrive at a comprehensive risk rating for each property. Using a formula or algorithm will ensure consistency in calculations. Further validating the scoring system with sample properties will help improve accuracy.

6. Validation and adjustment

It’s vital to validate and adjust the rating system against historical data and expert analysis to ensure its reliability and accuracy. Regularly review and update the criteria and weights based on new data.

Checklist

7. Providing risk mitigation recommendations

Along with the risk score, offering advice on how to reduce a property’s vulnerability to storm surges and hurricanes can be highly beneficial. Suggestions such as upgrading building materials, improving drainage systems or investing in flood barriers can form a checklist of actionable steps to reduce a property’s vulnerability.

8. Regular updates and re-evaluations

Continuously updating the risk rating system to reflect environmental changes, infrastructure developments and updated data is crucial. This includes regular reviews, incorporating new data and tech advancements can improve the risk rating system. 

Building Resilience with Accurate Risk Ratings

Stakeholders can create a robust and reliable risk rating system that enhances safety and preparedness in coastal areas.

A well-developed Storm Surge and Hurricane Risk Rating can provide essential information for making educated decisions about property development, insurance and risk management.

As the world grapples with the increasing challenges of climate change, these tools become ever more critical in our collective efforts to build resilient communities.

Subscribe to our newsletter

Subscribe to receive the latest blogs and data listings direct to your inbox.

Read more from The Proptech Cloud

Understanding Housing Affordability: Key Metrics and Statistics

Housing affordability is a significant concern in many parts of the world, affecting the quality of life and economic wellbeing of individuals and families.

To understand the dynamics of housing affordability we need to take a detailed look at a range of different metrics and statistics to gain a full picture.

Read more

The Three Primary Methods of Real Estate Data Integration

cLearn the three primary methods of real estate data integration—geospatial relationships, title matches, and address matching—to improve accuracy, insights, and decision-making.

Read more

What’s the Difference Between GDA94 and GDA2020?

Geodetic datums, or geodetic systems, are often used by proptechs. Here is a rundown of everything you need to know about the different geodetic datums we use and reference in Australia.

Read more

Alternative Data: What Is It, Who Uses It And Why It Matters

Discover the powerful intel alternative data can offer. Learn why businesses and investors are turning to non-traditional data sources for deeper insights and smarter decisions.

Read more

What Is A Geohash And How Is It Used?

Discover what a geohash is, how it works, and its real-world applications in mapping, logistics and data analysis.

Read more

How to Incorporate Mesh Blocks into Datasets

How to Incorporate Mesh Blocks into Datasets

May 28, 2024 | Technology Application

Mesh Blocks in real estate and proptech applications

Mesh Blocks are useful for geospatial and proptech applications, providing granularity and accuracy for understanding local real estate markets, demographics and land use.

The integration of Mesh Blocks into datasets can enhance the precision and relevance of analyses within the proptech and real estate sectors.

Useful in geospatial data and census analyses, embedding Mesh Blocks into digital boundaries enhances their usability in various applications.

We will cover the steps to incorporate mesh blocks into data sets below.

What are Mesh Blocks and how are they used in real estate?

Mesh Blocks are foundational building blocks for geospatial and proptech applications, providing granularity and accuracy for understanding local real estate markets, demographics and land use.

Learn more

How to incorporate Mesh Blocks into datasets

Incorporating Mesh Block into datasets involves several steps to ensure seamless integration and effective utilisation of geographical information. Here’s a guide on how to incorporate Mesh Blocks into datasets:

Step 1: Data Collection

Gather relevant data that aligns with Mesh Blocks.

This may include demographic information, property values, land use details, or any other dataset that can be associated with specific geographical areas.

 

Step 2: Download Mesh Block Boundaries

Mesh Block boundary files can be downloaded from authoritative sources, such as the Australian Bureau of Statistics (ABS) or relevant statistical agencies.

For ease, The Proptech Cloud has a free comprehensive dataset Geography – Boundaries & Insights – Australia ready for access and immediate use.

Geography – Boundaries & Insights – Australia

This free dataset from The Proptech Cloud is available for seamless access from Snowflake Marketplace.

Access data listing

Step 3: Geospatial Data Processing

Use Geographic Information System (GIS) software or programming libraries (e.g., Python with geospatial libraries like GeoPandas) to process and manipulate the mesh block boundaries.

Tip:

Geographical boundaries can be imported using Python libraries including Geopandas and shapely.

Many data warehouses including Snowflake, BigQuery and PostgreSQL have in-built geospatial functionality to allow for the processing of geospatial data.

QGIS – Loading in Geospatial files in QGIS

1. From the toolbar at the top of the page click Layer > Add Layer > Add Vector Layer

2. Make sure the Source Type is clicked to File

3. Load in the Source Data by using the three dots button at the side of the Vector Dataset(s) toolbar

QGIS - Loading in Geospatial files in QGIS

Geospatial Formats

The two most common ways geospatial data are represented in files are Well-Known-Text (WKT) which is a textual representation of a polygon and the geojson format which shows the coordinates and type of geojson format.

Both Python and Snowflake have capabilities to work with these 3 formats and parse them so they can be used in geography functions

WKT Format

#Example 2 using WKT format

from shapely import wkt

brisbane_bbox = “POLYGON ((153.012021 -27.471741, 153.012021 -27.462598, 153.032931 -27.462598, 153.032931 -27.471741, 153.012021 -27.471741))”

brisbane_poly = wkt.loads(brisbane_bbox)

Python – Loading in GeoJSON

The libraries geojson, shapely and json need to be installed.

#EXAMPLE 1 working with a geojson format

import json

import geojson

from shapely.geometry import shape

geojson_example = {

"coordinates": [[[153.01202116, -27.47174129], [153.01202116, -27.46259798], [153.03293092, -27.46259798], [153.03293092, -27.47174129], [153.01202116, -27.47174129]]],

"type": "Polygon"

}

geojson_json = json.dumps(geojson_example)

# Convert to geojson.geometry.Polygon

geojson_poly = geojson.loads(geojson_json)

poly = shape(geojson_poly ))

Snowflake

GeoJSON and WKT format can also be loaded into snowflake and converted to a geometry using the following commands:

#converting Well-Known-Text into geography format

SELECT ST_GEOGRAPHYFROMWKT('POLYGON ((153.012021 -27.471741, 153.012021 -27.462598, 153.032931 -27.462598, 153.032931 -27.471741, 153.012021 -27.471741))');

#Converting Geojson to geography format

SELECT TO_GEOGRAPHY('{

"coordinates": [[[153.01202116, -27.47174129], [153.01202116, -27.46259798], [153.03293092, -27.46259798], [153.03293092, -27.47174129], [153.01202116, -27.47174129]]],

"type": "Polygon"

}

')

Step 4: Data Matching

Match the dataset records with the appropriate mesh blocks based on their geographical coordinates. This involves linking each data point to the corresponding mesh block within which it is located.

Tip:

Geospatial functions which are supported in big data warehouses and Python can be used to match geospatial data.

A common way to match two geographical objects is to see if the coordinates of the two objects intersect. An example of how to do this in Python and Snowflake is shown below.

In Python

Data matching can be done using the shapely library intersects function.

from shapely import wkt, intersects

shape1 = wkt.loads("POLYGON ((153.012021 -27.471741, 153.012021 -27.462598, 153.032931 -27.462598, 153.032931 -27.471741, 153.012021 -27.471741))")

shape2 = wkt.loads("POLYGON ((153.012021 -27.471741, 153.032931 -27.462598, 153.012021 -27.471741))")

shape_int = intersects(shape1, shape2)

print(shape_int)

 

In Snowflake

Data matching can be done using the ST_Intersects function. One of the advantages of using big data warehouses including Snowflake to geospatially match data is that it leverages its highly scalable infrastructure to quickly complete geospatial processing.

WITH geog_1 as (

SELECT ST_GEOGRAPHYFROMWKT('POLYGON ((153.012021 -27.471741, 153.012021 -27.462598, 153.032931 -27.462598, 153.032931 -27.471741, 153.012021 -27.471741))') as poly

),

geog_2 as (

SELECT ST_GEOGRAPHYFROMWKT('POLYGON ((153.012021 -27.471741, 153.022021 -27.465, 153.032931 -27.462598, 153.012021 -27.471741))') as poly

)

SELECT

g1.poly, g2.poly

FROM geog_1 as g1

INNER JOIN geog_2 as g2

on ST_INTERSECTS(g1.poly, g2.poly)

Step 5: Attribute Joining

If your dataset and mesh blocks data have common attributes (e.g., unique identifiers), perform attribute joins to combine information from both datasets. This allows you to enrich your dataset with additional details associated with mesh blocks.

Step 6: Quality Assurance

Verify the accuracy of the spatial integration by checking for any discrepancies or errors. Ensure that each data point is correctly associated with the corresponding mesh block.

Tip:

geojson.io is a handy website that can help with visualising geojson data and ensure it is correct.

If you’re using Snowflake, the ST_AsGeojson command can be used to convert geography into a geojson which allows you to quickly visualise the shapes created.

Step 7: Data Analysis and Visualisation

Leverage the integrated dataset for analysis and visualisation. Explore trends, patterns, and relationships within the data at the mesh block level. Utilise geospatial tools to create maps and visual representations of the information.

Tip:

It’s worth mentioning that Snowflake has the option to create a Streamlit app within the Snowflake UI which allows for the cleaning and processing of data using Python and SQL and the interactive visualisation of data through the Streamlit App.

Read our blog which demonstrates how to predict migration patterns and create forecasts using Snowpark and Streamlit>

Snowflake also integrates really well with local Python development environments so all the initial data processing and cleaning can be done through a Snowflake API, then geography can be converted to a GeoJson or Text formal. Thereafter, libraries like plotly, folium, pydeck can be used to do complex geospatial visualisations.

Step 8: Data Storage and Management

Establish a system for storing and managing the integrated dataset, ensuring that it remains up-to-date as new data becomes available.

Consider using databases or platforms that support geospatial data.

Tip:

Geospatial datasets are usually very large and complex datasets due to the number of attributes included in a geospatial dataset, the resolution of the data and the number of records.

Cloud-based big data platforms can be an excellent option for storing geospatial data due to the low-cost of storage. Many of these platforms including also have spatial clustering options so that geospatial data in a similar location are grouped together, meaning queries for data in certain areas run more efficiently.

Snowflake (Enterprise Edition or Higher) also has an option to add a search optimisation to geospatial data tables to optimise the performance of queries that use geospatial functions.

Step 9: Documentation

Document the integration process, including the source of mesh block boundaries, any transformations applied, and the methods used for data matching. This documentation is essential for transparency and replicability.

By following these above steps, you can effectively incorporate mesh blocks into your datasets, enabling a more detailed and location-specific analysis of the information at the mesh block level.

 

Geography – Boundaries & Insights – Australia

This free dataset from The Proptech Cloud is available for seamless access from Snowflake Marketplace.

Access data listing

All rights are reserved, and no content may be republished or reproduced without express written permission from Data Army and The Proptech Cloud. All content provided is for informational purposes only. While we strive to ensure that the information provided here is both factual and accurate, we make no representations or warranties of any kind about the completeness, accuracy, reliability, suitability, or availability with respect to the blog or the information, products, services, or related graphics contained on the blog for any purpose.

Subscribe to our newsletter

Subscribe to receive the latest blogs and data listings direct to your inbox.

Read more from The Proptech Cloud

Understanding Housing Affordability: Key Metrics and Statistics

Housing affordability is a significant concern in many parts of the world, affecting the quality of life and economic wellbeing of individuals and families.

To understand the dynamics of housing affordability we need to take a detailed look at a range of different metrics and statistics to gain a full picture.

Read more

The Three Primary Methods of Real Estate Data Integration

cLearn the three primary methods of real estate data integration—geospatial relationships, title matches, and address matching—to improve accuracy, insights, and decision-making.

Read more

What’s the Difference Between GDA94 and GDA2020?

Geodetic datums, or geodetic systems, are often used by proptechs. Here is a rundown of everything you need to know about the different geodetic datums we use and reference in Australia.

Read more

Alternative Data: What Is It, Who Uses It And Why It Matters

Discover the powerful intel alternative data can offer. Learn why businesses and investors are turning to non-traditional data sources for deeper insights and smarter decisions.

Read more

What Is A Geohash And How Is It Used?

Discover what a geohash is, how it works, and its real-world applications in mapping, logistics and data analysis.

Read more

What Are Mesh Blocks & How Are They Used in Real Estate

What Are Mesh Blocks & How Are They Used in Real Estate

Mar 22, 2024 | Data Domain

In this article, we explore what Mesh Blocks are, how they are structured and their use in real estate and proptech.

Table of Contents

Jump ahead ↓

What are Mesh Blocks?

As defined by Australian Bureau of Statistics (ABS), mesh blocks are the smallest geographical area of the Australian Statistical Geography Standard (ASGS) and ABS’s classification of Australia into a hierarchy of statistical areas.

Mesh Blocks are essentially a set of geographic boundaries designed to segment Australia into very small areas. These boundaries are used to apply a systematic grid over the entire country, dividing it into tiny sections called Mesh Blocks.

Each Mesh Block is a polygon that outlines a specific piece of land, which can range from a single block in a city to a vast, sparsely populated area in the countryside.

In 2021, the ABS reported 368,286 Mesh Blocks covering the whole of Australia without gaps or overlaps.

Mesh Blocks covering the whole of Australia. Source: ABS Maps

 

Mesh Block design

Mesh Blocks for the current ASGS Edition 3 are designed according to a standard set of design criteria first developed for ASGS 2011.

Most Mesh Blocks are designed to contain 30 to 60 dwellings, although some low dwelling count Mesh Blocks exist. They are permitted in order to account for other design criteria.

The reasons for the minimum dwelling count of Mesh Blocks is so they’re small enough to aggregate to a wide range of areas, allow comparisons between geographic regions but also prevent accidentally exposing confidential information of individuals or businesses.

 

Mesh Block changes

Mesh Blocks are updated (or redesigned) every 5 years to stay relevant.

Mesh Blocks for the current ASGS Edition 3 was redesigned to ensure it still meets the design criteria first developed for ASGS 2011 and reflects the growth and change in Australia’s population, economy and infrastructure.

Mesh Block Changes

Example of Mesh Block change along the border of Queensland and New South Wales. Source: Australian Bureau of Statistics

How are Mesh Blocks created?

Each Mesh Block is assigned a unique numerical code or identifier. This code is used to reference the Mesh Block in statistical databases and geographic information systems (GIS).

The format of the code can vary but often includes digits that signify hierarchical levels of geography.

In Australia, Mesh Block identifiers are 11-digit codes.

The 11-digit Mesh Block code comprises: State and Territory identifier (1 digit), and a Mesh Block identifier (10 digits).

How are Mesh Blocks used?

The ABS does not and cannot provide detailed segmentation data (Census data) that can be directly connected to individuals or businesses. Instead, they provide anonymised and aggregated data against geographic areas. Mesh Blocks are the smallest geographic area that the ABS provide statistics against, so it offers population and dwelling counts at a hyper-local level – this is particularly useful for Census analysis.

These geographic boundaries allow for the aggregation of data from individual Mesh Blocks into larger geographic units, such as suburbs, towns, cities, and regions. This hierarchical structuring makes it possible to analyse data at various levels, from very detailed local information to broader regional or national trends.

Most businesses, including proptechs, looking to augment their analysis with population segmentation data will adopt Mesh Blocks as their default level geographic unit to gain the highest level of accuracy. The popularity of Mesh Blocks mean many businesses will use it for geographic statistics regardless of whether or not the Census data is being used.

What role do Mesh Blocks play in proptech?

Mesh Blocks play a vital role in Proptech, geospatial data, and the real estate industry in Australia. Some example uses include:

Granular geographical data

Since Mesh Blocks are the smallest geographical units, providing a granular level of detail in geographic data, its precision is valuable for analysing real estate trends at a hyper-local level.

Accurate small area statistics

Mesh Blocks are designed to fulfill the need for accurate small area statistics. In Proptech, having precise data at this level is instrumental for understanding localised property markets, demographics, and trends.

Spatial mapping and analysis

Geospatial data, including Mesh Blocks, facilitates spatial mapping and analysis. Proptech platforms can leverage this data to visualise and analyse property-related information, helping users make more informed decisions based on geographical insights.

Enhanced property valuation

Proptech applications can utilise Mesh Blocks to refine property valuation models. The data on dwellings and residents at this level allows for a more nuanced understanding of property values, considering localised factors.

Land use identification

Mesh blocks broadly identify land use, such as residential, commercial, industrial, parkland, and so forth. Land use information is valuable for proptechs involved in property development, urban planning, and investment strategies.

Targeted marketing and outreach

Proptech businesses can use Mesh Block data to tailor marketing and outreach strategies to specific geographical areas. Understanding the demographics and dwelling counts at this level allows for targeted and effective location-based campaigns.

Census-driven insights

The inclusion of Census data within Mesh Blocks, such as the count of usual residents and dwelling types, provides proptech platforms with up-to-date demographic information. This can aid market analysis, customer profiling, and investment strategies.

Integration with digital boundary files

The availability of Mesh Block boundaries in digital boundary files enhances their usability in Proptech applications. These files can be readily integrated into geospatial systems, making it easier for developers and analysts to work with this geographical data.

The foundational building blocks in real estate

Mesh Blocks are foundational building blocks for geospatial and proptech applications, providing granularity and accuracy for understanding local real estate markets, demographics and land use.

To aid proptechs, The Proptech Cloud offers its Geography – Boundaries & Insights dataset which includes all mesh blocks and their spatial areas for analysis and location-based visualisation of statistics.

The integration of this important information can enhance the precision and relevance of analyses within the proptech and real estate sectors. Read our following blog to learn how to incorporate Mesh Blocks into datasets.

How to Incorporate Mesh Blocks into Datasets

Incorporating mesh blocks into datasets involves several steps to ensure seamless integration and effective utilisation of geographical information. Here’s a guide on how to incorporate mesh blocks into datasets.

Learn more

Subscribe to our newsletter

Subscribe to receive the latest blogs and data listings direct to your inbox.

Read more from The Proptech Cloud

Understanding Housing Affordability: Key Metrics and Statistics

Housing affordability is a significant concern in many parts of the world, affecting the quality of life and economic wellbeing of individuals and families. To understand the dynamics of housing affordability we need to take a detailed look at a range of different metrics and statistics to gain a full picture.
Read more

The Three Primary Methods of Real Estate Data Integration

cLearn the three primary methods of real estate data integration—geospatial relationships, title matches, and address matching—to improve accuracy, insights, and decision-making.

Read more

What’s the Difference Between GDA94 and GDA2020?

Geodetic datums, or geodetic systems, are often used by proptechs. Here is a rundown of everything you need to know about the different geodetic datums we use and reference in Australia.

Read more

Alternative Data: What Is It, Who Uses It And Why It Matters

Discover the powerful intel alternative data can offer. Learn why businesses and investors are turning to non-traditional data sources for deeper insights and smarter decisions.

Read more

What Is A Geohash And How Is It Used?

Discover what a geohash is, how it works, and its real-world applications in mapping, logistics and data analysis.

Read more