Send the link below via email or IMCopy
Present to your audienceStart remote presentation
- Invited audience members will follow you as you navigate and present
- People invited to a presentation do not need a Prezi account
- This link expires 10 minutes after you close the presentation
- A maximum of 30 users can follow your presentation
- Learn more about this feature in our knowledge base article
Do you really want to delete this prezi?
Neither you, nor the coeditors you shared it with will be able to recover it again.
Make your likes visible on Facebook?
Connect your Facebook account to Prezi and let your likes appear on your timeline.
You can change this under Settings & Account at any time.
OSINT in 2013
Transcript of OSINT in 2013
August 29, 2013
Who is this presentation for?
kid stuff, right?
Entity extraction refers to the process of extracting structured information from unstructured data sources.
Such structured data may include:
Entities, events and documents
Properties associated with entities, events and documents
Relationships between entities, events and documents
In Palantir, entity extracted documents are represented in the DocXML format.
Finding Data Sources
Open-source intelligence (OSINT) is a form of intelligence
collection management t
publicly available sources
it to produce
Axciom - http://www.databyacxiom.com/
LexisNexis - http://www.lexisnexis.com/
ChoicePoint (now LexisNexis) - http://www.lexisnexis.com/
KnowX (now LexisNexis) - https://www.knowx.com/
DOCUSEARCH - http://www.docusearch.com/
Discreet Data - http://www.discreetdata.com/
MasterFiles - http://www.masterfiles.com/
InfoChimps - http://www.infochimps
Intelius - http://www.intelius.com
To perpetrate the scam, the fraudsters would set up fake mail boxes and then use information obtained on LexisNexis to open credit cards in the victims' names. The criminals were able to obtain names, dates of birth, and even Social Security numbers from the data broker.
OSS Data Sets
Data model is key
Connections are important
<Entity Type=’Phrase’><Value>Hello $entityValue</Value></Entity>
"Our policy clearly states that we 'provide information products which include financial information, Social Security number and other related information where permitted by law,' and that this information is 'provided to government agencies for the purposes of verifying information, employment screening and assisting law enforcement.'"
In 2003, over 1.6 billion customer records were stolen during the transmission of information to and from Acxiom's clients; the information included names, addresses, and email addresses. Acxiom's firewall was not breached, and no databases were accessed by the hackers. Prosecutors described the 2006 case against the hacker accused of stealing the data as the "largest ever invasion and theft of personal data" ever tried
Get rid of bogus data
Filter out interesting data
Depends on how you view the world
Java client app, can run local / remote
Great data analysis capabilities
Good integration / API / (TAS)
Can pull from database (SQLTAS)
Create a "Live search" - basic web service which performs scraping on the backend
Interacts with databases
Outputs to database, xml, csv, web services, etc
Integrates nicely with analysis tools
Made by Palantir
Deployed at Fortune 50 companies
Product is a child of PayPal
Incredible analysis platform
Logfiles, internal databases, OSINT sources
Traditional pentesting tools just find data
Data isn't the flippin problem
We're swimming (sinking) in data
Open and Closed sources ("Deep web")
Many OSINT source collections
Netblock or network
Module Based Tasks
Parent / Child relationships
Ruby on Rails
Omniture's Discover On Premise
A hierarchical type system of the real-world objects that human experts use to think about this problem. We call these PTObjects, short for “Palantir Objects”.
A type system of properties that will contain the data describing these PTObjects. PTObjects are essentially typed containers for properties. This is where most of the detail of the ontology lies.
A type system of possible relationships between different types of PTObjects.
The data sources are mapped into the ontology
The data are composed into real-world objects.
The server exposes Palantir “system calls”
Most of the time and effort in machine learning is spent getting the data into a form that you can actually apply an algorithm to!
It’s about the start of the analysis age
If we as a country understand what the danger is, then it's possible to fight this without giving up our civil liberties
Democracies tend to win when the people get behind it
What can we as citizens do (We as Americans) that allows us to experience the liberty we want, and stop them?
Educate the public about the threat
If we don't believe in what we're doing we can't win
Software has democratized espionage
LexisNexis Group provides computer-assisted legal research services. During the 1970s, LexisNexis pioneered the electronic accessibility of legal and journalistic documents. In 2006, the company had the world's largest electronic database for legal and public-records related information.
In 2000, LexisNexis purchased RiskWise, a St. Cloud, Minnesota company. In 2002 it acquired a Canadian research database company, Quicklaw. In 2004, Reed Elsevier Group, parent company of LexisNexis, purchased Seisint, Inc, of Boca Raton, Florida. Seisint housed and operated Multistate Anti-Terrorism Information Exchange (MATRIX).
According to a company news release, LexisNexis hosts over 30 terabytes of content on its 11 mainframes (supported by over 300 midrange UNIX servers and nearly 1,000 Windows NT servers) at its main datacenter in Miamisburg, Ohio.
DNS / Whois / Robtex
PublicData.com, et al
National property — Acxiom offers
the most comprehensive national
property database available in the
market today. Updated monthly,
national property includes 1,575
county assessor and 700 sales
records, making it the database of
choice for all real-property needs.
Comprehensive report — Acxiom
provides a detailed view of
information for a target individual.
The individual’s information
encompasses multiple sets of
data, which include the following:
the results from the Find People,
address history, alias, phone
history, relatives, associates (people
and corporations), concealed
weapons, licenses (drivers, vehicle,
hunting and fishing, professional
and pilot), property information,
and voter information
Acxiom’s combined information produces
some of the best hit and contact rates in
the industry, enabling you to consistently
recognize the consumer requested search
information even with name changes,
unreported moves or missing data
Acxiom leads the industry in fraud detection and identity verification of individuals with little to no credit history, enabling you to locate, recognize and verify individuals others cannot
Not for commercial use!
Maximum of 12 results per transform
You need to register on our website to use the client
API keys expire every couple of days
Runs on a (slower) server that is shared with all community users
Communication between client as server is not encrypted
Not updated until the next major version (and we know there are some bugs)
No end user support – you are on your own..
No updates of transforms on server side
Entity -> Transform -> Entities
Data Selection / Aquisition
Data Filtering / Analysis
nslookup / dig / whois
etc etc etc
Planning and Direction
Finding Data Sources
Transform Data -> Information
Transform Information -> Intelligence
OSINT in 2013
ZOMG SLIDE REMOVED
Public Records - brbpub.com/
Purchase Data Sets
Find and maintain relationships
OSINT In the context of Intelligence Gathering
Knowing more about prospects
Knowing more about your customers
Knowing more about your competitors
Knowing more about your business threats
- Leaked credentials
- Leaked business information
- Rogue employees
Finding upcoming malicious activity
Know what your attackers know
Write your own scripts
Right now in Conway, Ark., north of Little Rock, more than 23,000 computer servers are collecting, collating and analyzing consumer data for a company that, unlike Silicon Valley’s marquee names, rarely makes headlines. It’s called the Acxiom Corporation, and it’s the quiet giant of a multibillion-dollar industry known as database marketing.
In investor presentations and interviews, Acxiom executives have said that the company — the subject of a Sunday Business article last month — has information on about 500 million active consumers worldwide, with about 1,500 data points per person. Acxiom also promotes a program for consumers who wish to see the information the company has on them.
Several days later, Ms. Barrett Glasgow called to explain the delay in processing: Acxiom receives, on average, fewer than 100 requests a year from consumers, she said, and my check had “ended up on someone’s desk that was on vacation.” She said she would look into why company representatives hadn’t returned my voice mail message.
reverse phone search
Know what your attacker knows about you
Have an attacker mindset for your organization
Business Intelligence can use OSINT
OSINT is more than gathering flippin data manually
Data brokers are scary, need more control
Big Data presents new opportunities, problems
Check out Tapir