Stock Extractor

This package includes a series of stock data extractor class from a few widely used sources, such as Yahoo Finance, Barchart.com, etc.

Installation

pip install stock_extractor

The package has the following dependencies:

• requests
• pandas
• beautifulsoup4

USAGE

The package currently has four extractor class

• SP500Extractor

  It is used to extract stock data about S&P 500 companies

• YahooFinanceInfoExtractor

  It is used to extract latest stock data and information from Yahoo Finance

• YahooFinanceHistoryQuoteExtractor

  It is used to extract historical quote data for stocks from Yahoo Finance

• YahooFinanceDivExtractor

  It is used to extract historical dividend data for stocks from Yahoo Finance

SP500Extractor

# import extractor class
from stock_extractor import SP500Extractor

extractor = SP500Extractor()

# get_sp500_symbol_list() returns all SP500 company symbols as a list
sp500_symbols = extractor.get_sp500_symbol_list()

# get_sp500_data_by_type(type) retrieves sp500 company stock infomation and store the result in a pandas dataframe
# type can be 'main', 'technical', or 'performance'
# 'main' includes fields: 'Symbol', 'Name', 'Last Price', 'Change', 'Percent', 'High', 'Low', 'Volume', 'Time'
# 'technical' includes fields: 'Symbol', 'Name', 'Last Price', 'Opinion', '20D-Strength', '20D-Volty', '20D-AVol', '52W-Low', '52W-High'
# 'performance' includes fields: 'Symbol', 'Name', 'Last Price', 'Weighted-Alpha', 'YTD-Pct', '1Month-Pct Change', '3Month-Pct Change', '1Year-Pct Change'
extractor.get_sp500_data_by_type('technical')

# get_sp500_full_data() will retrieve all three types of data and combine them into the dataframe
extractor.get_sp500_full_data()

# get_dataframe() will return the dataframe that stores retrieved data
extractor.get_dataframe()

# save_to_csv(filepath) will save the dataframe as a csv file
extractor.save_to_csv('sp500_data.csv')

YahooFinanceInfoExtractor

This extractor class extract latest stock data and stock information on Yahoo Finance The fields that can be extracted from this class include:

'Ask', 'AvgDVol', 'AskSize', 'Bid', 'AskRealTime', 'BidRealTime', 'BookValue', 'BidSize', 'Change&Pct', 'Change', 'Commission', 'ChangeRealTime', 'AfterHourChangeRealTime', 'Dividend', 'LastTradeDate', 'TradeDate', 'EPS', 'ErrorIndication', 'EPSE_CurrentYear', 'EPSE_NextYear', 'EPSE_NextQuarter', 'FloatShares', 'D-Low', 'D-High', '52W-Low', '52W-High', 'HoldingsGainPercent', 'AnnualizedGain', 'HoldingsGain', 'HoldingsGainPercentRealTime', 'HoldingsGainRealTime', 'MoreInfo', 'OrderBookRealTime', 'MarketCap', 'MarketCapRealTime', 'EBITDA', 'ChangeFrom52W-Low', 'PctChangeFrom52W-Low', 'LastTradeRealTime', 'PctChangeRealTime', 'LastTradeSize', 'ChangeFrom52W-High', 'PctChangeFrom52W-High', 'LastTradeWithTime', 'LastTradePrice', 'HighLimit', 'LowLimit', 'DayRange', 'DayRangeRealTime', '50MA', '200MA', 'ChangeFrom200MA', 'PctChangeFrom200MA', 'ChangeFrom50MA', 'PctChangeFrom50MA', 'Name', 'Notes', 'Open', 'PreviousClose', 'PricedPaid', 'PctChange', 'Price/Sales', 'Price/Book', 'Ex-DividendDate', 'P/E', 'DividendPayDate', 'P/E_RealTime', 'PEG', 'P/E-EstCurrentYear', 'P/E-EstNextYear', 'Symbol', 'SharesOwned', 'ShortRatio', 'LastTradeTime', 'TradeLinks', 'TickerTrend', '1YrTarget', 'Volume', 'HoldingsValue', 'HoldingsValueRealTime', '52W-Range', 'DayValueChange', 'DayValueChangeRealTime', 'StockExchange', 'Yield'

Example:

# import extractor class
from stock_extractor import YahooFinanceInfoExtractor

extractor = YahooFinanceInfoExtractor()

# read a list of symbols from txt file.
# extractor.set_symbol_list(symbol_list) can set symbol list as python list
extractor.read_symbol_list_from_txt('../sample_data/sample_symbol_list.txt')

# set which fields are included in extraction
extractor.set_field_list([
        'Symbol', 'LastTradePrice', 'LastTradeDate', 'LastTradeTime', 'D-High', 'D-Low', '52W-High', '52W-Low',
        '50MA', '200MA', 'PctChangeFrom50MA', 'PctChangeFrom200MA', 'EBITDA', 'MarketCap',
        'Dividend', 'Yield', 'EPS', 'P/E', 'PEG', 'Price/Sales', 'Price/Book', 'Name'
])

# extract data from Yahoo Finance
extractor.load_yahoo_data()

# save the result in a csv file
# you can call extractor.get_dataframe() to return the result as pandas dataframe
extractor.save_to_csv('../output/sample_stock_info.csv')

YahooFinanceHistoryQuoteExtractor

This extractor extract historical quote data for the input symbol list for a time span

Example:

# import extractor class
from stock_extractor import YahooFinanceHistoryQuoteExtractor

extractor = YahooFinanceHistoryQuoteExtractor()

# set start and end date for extraction
extractor.set_end_date('2016-01-01')
extractor.set_start_date('2015-01-01')

# set symbol list, you can also set this from a txt file, see example above
extractor.set_symbol_list(['CAT', 'SPLS', 'ETP', 'HCP', 'T'])

# the method that actually extracts data from Yahoo Finance
extractor.load_data_by_symbol_list()

# filter out other fields, leave only adj price here since this field is usually what people need.
extractor.get_adj_price_only_dataframe()

#return the result as a pandas dataframe
extractor.get_dataframe()

# save the result as csv file
extractor.save_to_csv()

YahooFinanceDivExtractor

This extractor extracts historical dividend data from Yahoo Finance. The api is very similar to the previous one.

Example:

# import extractor class
from stock_extractor import YahooFinanceDivExtractor

extractor = YahooFinanceDivExtractor()
extractor.set_end_date('2016-01-01')
extractor.set_start_date('2001-01-01')
extractor.read_symbol_list_from_txt('../sample_data/sample_symbol_list.txt')
extractor.load_data_by_symbol_list()
extractor.get_dataframe()
extractor.save_to_csv()

TEST

run this code

$ python -m unittest discover

CONTACT

The package is created by Zach Liu. Please send email to zachliugis@gmail.com if you have questions or comments.

LICENCE

MIT