APWG Applied Research

Counter-Cybercrime Innovation in the Public Interest - Since 2003

APWG’s research and development programs establish data assets, metrics and conventions that are deployed as permanent working counter-cybercrime resources, such as the APWG eCrime eXchange cybercrime event data clearinghouse; the education-page redirect program, and the STOP. THINK. CONNECT. cybersecurity awareness campaign.

The Applied Research Secretariat’s imperative is to develop and deploy universally applicable resources to inform and optimize the global cybercrime-suppression infrastructure in order to neutralize cybercrime programmatically. This organizing principle reflects the criterion by which its R&D programs are prioritized: their potential scale for suppressing cybercrimes in a public-health model of intervention.


The criterion by which APWG's R&D programs are prioritized is their potential scale for suppressing the cybercrimes against which they are targeted in a public-health model of intervention

The APWG’s initial phishing URL repository was architected in Fall 2003 by Internet infrastructure managers, major bank security teams and security product developers – and deployed within months. Its antecedents are still operating to this day as components of the APWG eCrime eXchange, APWG’s cybercrime data clearinghouse, delivering billions of data elements to APWG members and helping to secure billions of devices and software clients.

Likewise, APWG’s Phishing Education Landing Page was conceived by Reseachers from Carnegie-Melon University presenting at the APWG 2007 eCrime Conference, for instance. The landing page was operational within a few months, now educating each month hundreds of thousands of credulous users who click on links to decommissioned phishing websites – and responding to them in some 21 languages.

APWG’s proposal for a messaging convention of commercial enterprises and governments sharing a common cybersecurity campaign of shared media assets, was first proffered to its members in winter of 2008. That proposal inspired and informed the development of the STOP. THINK. CONNECT. campaign and its adoption by the US Government in 2010. The campaign has subsequently been launched in 22 other nations. Meanwhile, dozens of other national ministries and NGOs, in addition, have signed the Messaging Convention’s memorandum of cooperation in preparation for their own campaign launches.

APWG R&D Process Flow
The cybercrime research community converged on APWG after it established the Phishing URL Block List in 2004, providing investigators in academia and industry with a consistent, relevant data schema for phishing attack data with which they could inform analyses and experiments. Those researchers' quickly moved APWG to found its peer-reviewed research conference, APWG eCrime. Over the years, the research community and APWG's institutional cohorts have guided our R&D efforts, sometimes insisting the institution develop resources that are neither product, nor service but essential for the global management of contemporary and evolving cybercrime

The applied research projects that APWG has completed and currently curates have often been provoked by research presented at the annual APWG Symposium on Electronic Crime Research, almost since the conference’s founding in 2006.  APWG’s applied research programs have also, however, reflected a broad mix of the interests of its members, the research community that coalesced around the eCrime conference, as well  the law enforcement community and civil society and intergovernmental organizations with which the APWG is regularly correspondent. Those programs have fallen broadly into three topical research topic sections: data logistics and telemetry; behavioral and neurocognitive dimensions of cybercrime victimization; and industrial and public policy aspects of cybercrime management. As follows:

The Data Logistics and Telemetry section engages questions of orchestrating the deployment of event data exchange at scale across computing platforms to effect more unified mutualistic responses to predictable cybercrimes like phishing.

Response to cybercrime has been a folklorish enterprise to date, organized principally around deployment of products and services to prevent, detect, remediate and investigate cybercrimes against enterprises and their brands and counter-parties. Programmatic data exchange, the lifeblood of all public-health regimens, has only been formally organized in cyber to a limited degree, denying the domain the kinds of efficiencies that attend rigorously curated public-health modalities of intervention.

APWG's Crypto Currency Working Group's Wallet Address Data Corpus Project

The APWG’s Crypto Currency Working Group’s Wallet Address Data Corpus Project hosts hundreds of thousands of addresses and adds thousands of new addresses each month to the database

The APWG eCrime eXchange

The APWG eCrime eXchange is the world’s largest NGO-managed clearinghouse for cybercrime-related machine event data, delivering upwards of billions of data elements per month outbound to its member institutions from industry, national governments and multi-lateral treaty organizations

The APWG PhishFarm Block List Latency Monitoring Program

The APWG PhishFarm Block List Latency Monitoring Program is designed to measure latency of updating of browser block lists. The principal objectives are to inform whole-of-ecosystem metrics to cultivate efficacious data logistics; to drive out control failures; and to measure subsequent ecosystem performance changes, the same way health agencies employ metrics to manage disease propagation.

Data/Telemetry Research Fellows
Brad Wardman
Brad Wardman, APWG Director Program Principal Investigator PHISHFARM
Adam Oest / PayPal
Adam Oest Program Principal Investigator PHISHFARM
Bernhard Haslhofer
Bernhard Haslhofer Austrian Institute of Technology Moderator CRYPTO CURRENCY WORKING GROUP
Arghya Mukherjee
Arghya Mukherjee APWG Curator Fellow CCWG Data Corpus Project Ph.D Candidate University of Tulsa
Emad Badawi UOttawa PhD Candidate Curator Fellow APWG
Emad Badawi APWG Curator Fellow CCWG Data Corpus Project UOttawa PhD Candidate
Patrick Cain, APWG
Patrick Cain Resident Research Fellow APWG Cooper-Cain Group

To advance programmatic data exchange as a conventionalized discipline, APWG has stepped forward to develop programs such as: PhishFarm, a browser block list latency monitoring program to measure efficiencies of block lists in updating the URLs they deflect users from visiting; and the Crypto Currency Working Group Data Corpus Project, a program to fuse data from sources of wallet addresses associated with common cybercrimes such as ransomware and bitcoin generator scams.

The development of mutualistic data provisioning schemes, conventionalized metrics for ecosystem performance measurement and management, as well as APWG’s curated data clearance and telemetry resources will promote establishment of common operational vocabulary for stakeholders to use to orchestrate and optimize a globalized cybercrime response ecosystem and mitigate control gaps discovered to be impairing response-infrastructure performance.

APWG’s research and development programs establish data assets, metrics and conventions that are deployed as permanent working counter-cybercrime resources for stakeholders the world over

The Human Factors section probes the dimensions of human responses to common cybercrime events and the perceptions, conditioning and knowledge that redound to enhance – or reduce – user resilience to those cybercrimes.

Every large-scale study of cybercrime either concludes or notes that user error  is involved in 95 percent or more of all cybercrime events like commercial data breaches and, for all the observers in the field, it is a matter of faith that cybercrime’s success is partly due to the built computing environment conditioning the user to be more easily manipulated by cybercriminals.

From the first year of APWG’s Symposium on Electronic Research in 2006 and every year after, a substantial proportion of the submissions to the review committee addresses human response aspects of cybercrime, even thought in that first year, the CFP did not include specific requests for behavioral aspects research.

The APWG’s Crypto Currency Working Group’s Wallet Address Data Corpus Project hosts hundreds of thousands of addresses and adds thousands of new addresses each month to the database


The Messaging Convention was proposed formally by APWG to its members in 2009. The subsequent STOP. THINK. CONNECT. campaign was adopted by the US government in 2010. Since then, the campaign has been launched by cabinet ministries and NGOs in another 22 other nations. Today, the STOP. THINK. CONNECT. Messaging Convention manages the campaign’s intellectual property and global footprint development.

APWG Phishing Education Landing Page Redirect Program

APWG’s Phishing Education Landing Page was conceived by Reseachers from Carnegie-Melon University presenting at the APWG 2007 eCrime Conference. The landing page was operational within a few months, now educating each month hundreds of thousands of credulous users who click on links to decommissioned phishing websites – in some 21 languages.

Human Factors Research Fellows
Paul Watters
Paul Watters Cyberstronomy

Many of those papers and the dialogs they’ve provoked, however, have moved APWG to organize important Applied Research programs that have established permanent cybercrime response and prevention resources such as: the Phishing Education Landing Page, a redirect system that ISPs can user to shunt users who’ve clicked on links to decommissioned phishing pages to an educational resource page; and the STOP. THINK. CONNECT. cybersecurity awareness campaign that has been launched as the US government’s own in 2010 and subsequently adopted and launched in more than 20 other nations.

The first eCrime research conference in 2006 surprised organizers with the proportion of papers focusing on behavioral aspects of cybercrime, inspiring APWG's first cybercrime awareness and education programs - all of which are still operating today worldwide


The Industrial and Public Policy section examines the role of law, regulation and industrial convention in mobilizing – and impeding – the broader response and management of common cybercrimes that menace all Internet users.

Products and services provide key protections and resources for securing cyberspace – but when everyone is fighting the same fires; or the same maritime pirates; or the same measles; civilizations organize clearinghouses, conventions and law for common defense. As a trade association, APWG finds pride in assisting industry in informing the decisions and applications of commercial enterprises. As well, APWG honors in memory the historical contributions of non-profit trade associations, government safety agencies, multilateral treaty organizations and standards bodies in providing keystone policy instruments to manage predictable risks programmatically over the centuries.

APWG has engaged data policy questions historically in a three dimensional approach:

  • APWG Data Policy SymposiumPresentation of original policy research: APWG opened up the CFP to its annual eCrime conference to policy studies submissions and organized a unique Data Policy Symposium which holds meetings for stakeholders in the United States and the EU to consider, for example, policy and regulatory impediments to data exchange vital to forensic and security applications employed by industry and law enforcement. (SEE: Policy and Position Papers, below);
  • Development of applied policy management instrumentation like the APWG’s Data UserAPWG Malicious Domain Suspension ProgramAgreement for eCX users (to manage liabilities attendant cybercrime data exchange) and policy-driven cybercrime response applications such as AMDoS, an affidavit delivery system for Sponsoring Registrars to receive reports of malicious domain name registrations from Accredited Reporters;
  • Contribution of policy analyses and proposals: offering expert-witness commentary to a number of Policy Section of APPLIED RESEARCH Secretariat multilateral treaty organizations (by invitation, for example, in the UN as recognized by the Doha Declaration and the Salvador Declaration) and governance and trade groups. [SEE Papers and Correspondence archive below.]

Over the years, APWG and its directors and research fellow have been called upon to provide commentary and presentations to the  United Nations (Office on Drugs and Crime), Organization for Security and Cooperation in Europe, Council of Europe’s Convention on CybercrimeEuropol EC3 the Organization of American States, the Commonwealth of Nations, the Commonwealth Parliamentary AssociationOrganisation for Economic Co-operation and DevelopmentInternational Telecommunications Union and ICANN; the European Commission, the G8 High Technology Crime Subgroup. APWG was a founding member of the steering group of the Commonwealth Cybercrime Initiative at the Commonwealth of Nations.

One of APWG’s technical diplomacy objectives – key to our applied policy efforts – is for the operational realties that industry manages to be fully and accurately considered in development of cybercrime law, regulation and policy. In practical terms, this requires addressing conflicts between them, and cultivating policy makers’ understanding of those operational aspects. Some of APWG’s policy papers and submissions to trade groups and treaty organizations follow:

Policy Research Fellows
Patrick Cain, APWG
Patrick Cain Resident Research Fellow APWG Cooper-Cain Group
Jesse Sowell - Texas A&M
Dr. Jesse Sowell Texas A&M
Greg Aaron, Illumintel / APWG
Greg Aaron Senior Research Fellow APWG
APWG Policy and Position Papers and Correspondence 2010 to 2020 (Abridged)
One of APWG's technical diplomacy objectives is for the operational realties that industry manages every day fighting cybercrime to be considered fully in the development of cybercrime law, regulation and policy

Commentary for the Sixth Meeting of the Intergovernmental Expert Group on Cybercrime / July 27 – 29 2020 in Vienna

APWG reviews its proposals for the United Nations Intergovernmental Group on Cybercrime to animate far more programmatic responses to common cybercrimes, including a universal nomenclature for cybercrime data; specific legal authority for private sector interveners to handle machine event data; and a conventionalized legal definition of Machine Event Data that would be poised to highlight the borders of PII.


Correspondence to ICANN Org from APWG’s Secretary General on behalf of the APWG Board of Directors. addressed to ICANN CEO Mr. Göran Marby Delivery via email April 5, 2018

APWG voices support for a tiered access scheme for qualified parties to maintain access to non-public WHOIS data after the initial roll out go the GDPR, citing the basic scheme of the Model 1.3 accreditation plan, known within the ICANN community as the ‘Cannoli Model’


United Nations Office on Drugs and Crime December 2010 Fifth meeting of the Core Group of Experts on Identity-Related Crime United National International Center 6-8 December 2010, Vienna, Austria SEE: Page, 19, Paragraph 48, 49 and 50.

APWG introduces cybercrime response utilities provided to industry, governments and civil sector actors to educate users exposed to cybercrime. In example, APWG reviews the APWG Phishing Education Fax-Back Page that instruct consumers about protecting themselves against offline phishing scams at the “most teachable moment”: when they have just responded to a phishing communication via fax. Here, too, APWG details the practical impediments can put in the way of evidentiary data between private sector responders and public agency law enforcement.


Cybercrime Convention Committee (T-CY) Public Hearing on Transferred Access to Data Written contributions Council of Europe 3 June 2013, Strasbourg, France (Page 3)

APWG addresses questions posted by the T-CY regarding the interpretation of Article 32b Operational Aspects of the Budapest Convention on Cybercrime of 2001 and other aspects of the convention on behalf of the Cybercrime Convention Committee and considers a posits a definition of machine event data as a necessary


Fourth meeting of the Core Group of Experts on Identity-related Crime  (Vienna, Austria, 18-22 January 2010)  SEE: Page 19, Paragraph 59

APWG reviews resources that its URL Block List provides to its members in industry, law enforcement and other public sector entities. APWG also details the Phishing Education Landing Page with Core Group of Experts, a redirect system that was then recently launched, automatically directing users clicking on links to decommissioned phishing sites to educational and awareness at the moment of potential misadventure.


PhishFarm: Block List Latency Monitoring

The APWG PhishFarm Block List Latency Monitoring Program would provide insights into the browser block lists that are one of the last lines of defense between workaday users and phishing websites — an enterprise that is essential to understanding the efficiency of the cybercrime response ecosystem. Operationally, we’ll be building a latency measurement scheme into the APWG eCrime eXchange that provides insight into the time lag between submitting a URL to eCX and the moment the block lists begin actually blocking the URL – if at all.

With a reliable system for measuring the latency and efficacy of block list updating stakeholders will have foundational metrics required for:

* Measurement of blocklist report instantiation efficiency
* Development of optimization schemes and policies for phishing URL reporting
* Discovery of sophisticated, high-impact attacks
* Measurement of mitigation efficacy
* Whole ecosystem performance assessment.

SEE:  https://ecrimeresearch.org/phishfarm/

National Cyber Resilience Baselining

The APWG is working with research centers in Australia, and the United States to deploy the world’s first national base-lining survey of user resilience to the common cybercrime of phishing to gain insights into behavioral aspects of phishing – and to establish data corpora for university and industry investigators researching the behavioral/cognitive dimensions of cybercrime.

Principle investigators from La Trobe University and Indiana University are organizing this program to engage user behaviour in cyber security as a public health problem, adapting techniques from epidemiology to generate data that is representative of the whole population – not a biased sub-sample. The data generated from this study will help to extend the field, but more importantly, will be shared with system designers to help build more secure tools and better incident response capability.

This study will extend an existing instrument developed at Indiana University that measures responses to simulated phishing attacks, and deliver it to a target 9,798 randomly sampled users nationally (approximately 0.2% of the population). This sample size has been selected because it is the minimum sample size required to achieve a Confidence Level of 0.99, with a Confidence Interval of 0.5, given a population of 24,511,800 in Australia.

Meanwhile. APWG is working with principal investigators in a number of European countries to consider the potential for deploying baselining studies of their nations’ populations as well.

SEE: https://ecrimeresearch.org/ncrb/

Crypto Currency Wallet Address Data Corpus

In 2018, the APWG inaugurated the APWG Crypto Currency Working Group (CCWG) to help cryptocurrency exchanges, wallet hosters, trading platforms and investment funds protect themselves and their customers against phishing and cybercrime – and established a data endpoint on the APWG’s eCrime eXchange (eCX) for wallet addresses associated with cybercrime events.

Today, the CCWG’s /crypto API endpoint on the APWG’s eCrime eXchange is delivering hundreds of millions of data entities per month outbound to its members, providing event records in a complete and verbose schema that provides key primary wallet address data for those payment instruments suspected of providing cash out mechanisms for scams and racketeering operations.

In development of the CCWG’s data corpus of wallet addresses, the APWG has turned to the research community that has formed up around the APWG’s Symposium on Electronic Crime Research (APWG eCrime) to mine for fresh insight into cybercrime’s unique challenges and for opportunities in sourcing event data for the CCWG.DB. Actively updated data is drawn from a number of online resources and experimental platforms – including, for example, a University of Ottawa recruitment platform that isolates bitcoin generator scams and writes the cash-out wallet addresses to the CCWG.DB.

SEE: https://ecrimeresearch.org/membership/ccwg/

  • ECRIME 2018 – SAN DIEGO, CA – MAY 15, 16 & 17 For 2018 APWG’s members will once again come together to bridge the gaps between cybersecurity operations, research and consumer messaging with our thirteenth Symposium on Electronic Crime Research (eCrime 2018). eCrime 2018 will......

  • For 2019 APWG's members will once again come together to bridge the gaps between cybersecurity operations, research and consumer messaging with our fourteenth Symposium on Electronic Crime Research (eCrime 2019)....

  • La trigésimo quinta edición del Congreso, SEGURINFO, promete continuar con sus aportes a la comunidad en un ámbito de creciente importancia a nivel mundial, como lo es la seguridad de la información. La intención de este evento, que organiza USUARIA desde el año 2005, es seguir......