File: //lib/python2.7/site-packages/bs4/__init__.pyo
�
qap[c @ s| d Z d Z d Z d Z d Z d g Z d d l Z d d l Z d d l Z d d l Z d d l
Z
d d l m Z m
Z
d d
l m Z d d l m Z m Z m Z m Z m Z m Z m Z m Z m Z m Z m Z d d
k d e f d � � YZ e Z e Z d e f d � � YZ d e f d � � YZ! d e" f d � � YZ# e$ d k rxd d l Z e e j% � Z& e& j' � GHn d S( sH Beautiful Soup
Elixir and Tonic
"The Screen-Scraper's Friend"
http://www.crummy.com/software/BeautifulSoup/
Beautiful Soup uses a pluggable XML or HTML parser to parse a
(possibly invalid) document into a tree representation. Beautiful Soup
provides methods and Pythonic idioms that make it easy to navigate,
search, and modify the parse tree.
Beautiful Soup works with Python 2.7 and up. It works better if lxml
and/or html5lib is installed.
For more than you ever wanted to know about Beautiful Soup, see the
documentation:
http://www.crummy.com/software/BeautifulSoup/bs4/doc/
s* Leonard Richardson (leonardr@segfault.org)s 4.6.3s* Copyright (c) 2004-2018 Leonard Richardsont MITt
BeautifulSoupi����Ni ( t builder_registryt ParserRejectedMarkup( t
UnicodeDammit( t CDatat Commentt DEFAULT_OUTPUT_ENCODINGt Declarationt Doctypet NavigableStringt PageElementt ProcessingInstructiont ResultSett SoupStrainert Tags` You are trying to run the Python 2 version of Beautiful Soup under Python 3. This will not work.su You need to convert the code, either by installing it (`python setup.py install`) or by running 2to3 (`2to3 -w bs4`).c B s e Z d Z d Z d d g Z d Z d Z d d d d d d d � Z d � Z d � Z
e d
� � Z d � Z
d � Z d d i d
� Z e d � Z d � Z d � Z d � Z d � Z e d � Z d d d � Z d e d � Z d � Z d d � Z d � Z e e d d � Z RS( s
This class defines the basic interface called by the tree builders.
These methods will be called by the parser:
reset()
feed(markup)
The tree builder may call these methods from its feed() implementation:
handle_starttag(name, attrs) # See note about return value
handle_endtag(name)
handle_data(data) # Appends to the current data node
endData(containerClass=NavigableString) # Ends the current data node
No matter how complicated the underlying parser is, you should be
able to build a tree using 'start tag' events, 'end tag' events,
'data' events, and "done with data" events.
If you encounter an empty-element tag (aka a self-closing tag,
like HTML's <br> tag), call handle_starttag and then
handle_endtag.
u
[document]t htmlt fasts
s� No parser was explicitly specified, so I'm using the best available %(markup_type)s parser for this system ("%(parser)s"). This usually isn't a problem, but if you run this code on another system, or in a different virtual environment, it may use a different parser and behave differently.
The code that caused this warning is on line %(line_number)s of the file %(filename)s. To get rid of this warning, pass the additional argument 'features="%(parser)s"' to the BeautifulSoup constructor.
t c s� d � k r t j d � n d � k r? � d =t j d � n d � k rb � d =t j d � n d � k r� � d =t j d � n d � k r� � d =t j d
� n � f d � } | p� | d d
� } | p� | d d � } | r t | t � r t j d � d) } n t � � d k rC� j � j � } t d | � � n | d) k r�| }
t | t � rp| g } n | d) k s�t | � d k r�| j
} n t j | � } | d) k r�t
d d j | � � � n | � } |
| j k p�|
| j k s�| j r
d } n d } d) }
y t j d � }
Wn t k
r<n X|
rX|
j } |
j } n t j } d } | j d � } | r�| j � } | j d* � r�| d } q�n | r�t d | d | d | j d | � } t j | j | d d! �q�q�n | | _ | j | _ | j | _ | | j _ | | _ t! | d" � rH| j" � } n� t | � d# k rAt | t# � rud$ | k s�t | t � rAd% | k rAt | t � r�t$ j% j& r�| j' d& � } n | } t( } y t$ j% j) | � } Wn t* k
r�} n X| r1t | t � r | j' d& � } n t j d' | � n | j+ | � n xh | j j, | | d( | �D]K \ | _- | _. | _/ | _0 | j1 � y | j2 � PWq]t3 k
r�q]Xq]Wd) | _- d) | j _ d) S(+ s_ Constructor.
:param markup: A string or a file-like object representing
markup to be parsed.
:param features: Desirable features of the parser to be used. This
may be the name of a specific parser ("lxml", "lxml-xml",
"html.parser", or "html5lib") or it may be the type of markup
to be used ("html", "html5", "xml"). It's recommended that you
name a specific parser, so that Beautiful Soup gives you the
same results across platforms and virtual environments.
:param builder: A specific TreeBuilder to use instead of looking one
up based on `features`. You shouldn't need to use this.
:param parse_only: A SoupStrainer. Only parts of the document
matching the SoupStrainer will be considered. This is useful
when parsing part of a document that would otherwise be too
large to fit into memory.
:param from_encoding: A string indicating the encoding of the
document to be parsed. Pass this in if Beautiful Soup is
guessing wrongly about the document's encoding.
:param exclude_encodings: A list of strings indicating
encodings known to be wrong. Pass this in if you don't know
the document's encoding but you know Beautiful Soup's guess is
wrong.
:param kwargs: For backwards compatibility purposes, the
constructor accepts certain keyword arguments used in
Beautiful Soup 3. None of these arguments do anything in
Beautiful Soup 4 and there's no need to actually pass keyword
arguments into the constructor.
t convertEntitiess� BS4 does not respect the convertEntities argument to the BeautifulSoup constructor. Entities are always converted to Unicode characters.t
markupMassages� BS4 does not respect the markupMassage argument to the BeautifulSoup constructor. The tree builder is responsible for any necessary markup massage.t
smartQuotesTos� BS4 does not respect the smartQuotesTo argument to the BeautifulSoup constructor. Smart quotes are always converted to Unicode characters.t selfClosingTagss� BS4 does not respect the selfClosingTags argument to the BeautifulSoup constructor. The tree builder is responsible for understanding self-closing tags.t isHTMLs� BS4 does not respect the isHTML argument to the BeautifulSoup constructor. Suggest you use features='lxml' for HTML and features='lxml-xml' for XML.c s< | � k r8 t j d | | f � � | } � | =| Sd S( NsL The "%s" argument to the BeautifulSoup constructor has been renamed to "%s."( t warningst warnt None( t old_namet new_namet value( t kwargs( s0 /usr/lib/python2.7/site-packages/bs4/__init__.pyt deprecated_argument� s
t parseOnlyTheset
parse_onlyt fromEncodingt
from_encodingsl You provided Unicode markup but also provided a value for from_encoding. Your from_encoding will be ignored.i s2 __init__() got an unexpected keyword argument '%s'sj Couldn't find a tree builder with the features you requested: %s. Do you need to install a parser library?t ,t XMLt HTMLi t __file__s .pycs .pyoi����t filenamet line_numbert parsert markup_typet
stackleveli t readi t <u <t utf8sw "%s" looks like a filename, not markup. You should probably open this file and pass the filehandle into Beautiful Soup.t exclude_encodingsN( s .pycs .pyo(4 R R t
isinstancet unicodeR t lent keyst popt TypeErrort
basestringt DEFAULT_BUILDER_FEATURESR t lookupt FeatureNotFoundt joint NAMEt ALTERNATE_NAMESt is_xmlt syst _getframet
ValueErrort f_globalst f_linenot __dict__t gett lowert endswitht dictt NO_PARSER_SPECIFIED_WARNINGt buildert known_xmlt soupR! t hasattrR- t bytest ost patht supports_unicode_filenamest encodet Falset existst Exceptiont _check_markup_is_urlt prepare_markupt markupt original_encodingt declared_html_encodingt contains_replacement_characterst resett _feedR ( t selfRX t featuresRJ R! R# R0 R R t argt original_featurest
builder_classR+ t callert globalsR) R( t fnlt valuest possible_filenamet is_filet e( ( R s0 /usr/lib/python2.7/site-packages/bs4/__init__.pyt __init__X s� '
#
.
c C s: t | � | j d � d | j d d �} | j | _ | S( Ns utf-8RJ R# ( t typeRR RJ RY ( R^ t copy( ( s0 /usr/lib/python2.7/site-packages/bs4/__init__.pyt __copy__$ s ! c C s9 t | j � } d | k r5 | j j r5 d | d <n | S( NRJ ( RH RD RJ t picklableR ( R^ t d( ( s0 /usr/lib/python2.7/site-packages/bs4/__init__.pyt __getstate__0 s
c s� t � t � r d } d } n" t � t � r< d } d
} n d St � f d � | D� � r� | � k r� t � t � r� � j d d
� } n � } t j d | � q� n d S( s�
Check if markup looks like it's actually a url and raise a warning
if so. Markup can be unicode or str (py2) / bytes (py3).
t s http:s https:u u http:u https:Nc 3 s | ] } � j | � Vq d S( N( t
startswith( t .0t prefix( RX ( s0 /usr/lib/python2.7/site-packages/bs4/__init__.pys <genexpr>F s s utf-8t replaces� "%s" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.( s http:s https:( u http:u https:( R1 RN R2 t anyt decodeR R ( RX t spacet cant_start_witht decoded_markup( ( RX s0 /usr/lib/python2.7/site-packages/bs4/__init__.pyRV 7 s c C sT | j j � | j j | j � | j � x# | j j | j k rO | j � q- Wd S( N( RJ R\ t feedRX t endDatat
currentTagt namet
ROOT_TAG_NAMEt popTag( R^ ( ( s0 /usr/lib/python2.7/site-packages/bs4/__init__.pyR] S s
c C sg t j | | | j | j � d | _ | j j � g | _ d | _ g | _ g | _
| j | � d S( Ni ( R Rj RJ R t hiddenR\ t current_dataR R} t tagStackt preserve_whitespace_tag_stackt pushTag( R^ ( ( s0 /usr/lib/python2.7/site-packages/bs4/__init__.pyR\ ] s
c K s) | j | � t d | j | | | | � S( s+ Create a new tag associated with this soup.N( t updateR R RJ ( R^ R~ t namespacet nsprefixt attrst kwattrs( ( s0 /usr/lib/python2.7/site-packages/bs4/__init__.pyt new_tagg s
c C s
| | � S( s7 Create a new NavigableString associated with this soup.( ( R^ t st subclass( ( s0 /usr/lib/python2.7/site-packages/bs4/__init__.pyt
new_stringl s c C s t d � � d S( Ns4 BeautifulSoup objects don't support insert_before().( t NotImplementedError( R^ t successor( ( s0 /usr/lib/python2.7/site-packages/bs4/__init__.pyt
insert_beforep s c C s t d � � d S( Ns3 BeautifulSoup objects don't support insert_after().( R� ( R^ R� ( ( s0 /usr/lib/python2.7/site-packages/bs4/__init__.pyt insert_afters s c C s^ | j j � } | j r; | | j d k r; | j j � n | j rW | j d | _ n | j S( Ni����( R� R5 R� R} ( R^ t tag( ( s0 /usr/lib/python2.7/site-packages/bs4/__init__.pyR� v s c C sk | j r | j j j | � n | j j | � | j d | _ | j | j j k rg | j j | � n d S( Ni����( R} t contentst appendR� R~ RJ t preserve_whitespace_tagsR� ( R^ R� ( ( s0 /usr/lib/python2.7/site-packages/bs4/__init__.pyR� s c C s� | j r� d j | j � } | j s{ t } x' | D] } | | j k r1 t } Pq1 q1 W| r{ d | k ro d } qx d } q{ n g | _ | j r� t | j � d k r� | j j s� | j j
| � r� d S| | � } | j | � n d S( Nu s
Rq i ( R� R; R� t Truet ASCII_SPACESRS R! R3 R� t textt searcht object_was_parsed( R^ t containerClassR� t
strippablet it o( ( s0 /usr/lib/python2.7/site-packages/bs4/__init__.pyR| � s&
c C s� | p | j } | p | j } d } } } t | t � rk | j } | j } | j } | sk | j } qk n | j | | | | | � | | _ | j
j | � | j r�t | j
� d } xG | d k r� | j
| | k r� Pn | d 8} q� Wt
d | | f � � | d k r | } d } n | j
| d } } | t | j
� d k r^| j } d } n | j
| d } } | | _ | r�| | _ n | | _ | r�| | _ n | | _ | r�| | _ n | | _ | r�| | _ q�n d S( s Add an object to the parse tree.i i s[ Error building tree: supposedly %r was inserted into %r after the fact, but I don't see it!N( R} t _most_recent_elementR R1 R t next_elementt next_siblingt previous_siblingt previous_elementt setupR� R� R3 RA ( R^ R� t parentt most_recent_elementR� R� R� R� t index( ( s0 /usr/lib/python2.7/site-packages/bs4/__init__.pyR� � sR c C s� | | j k r d Sd } t | j � } xn t | d d d � D]V } | j | } | | j k r� | | j k r� | r� | j � } n Pn | j � } q? W| S( s� Pops the tag stack up to and including the most recent
instance of the given tag. If inclusivePop is false, pops the tag
stack up to but *not* including the most recent instqance of
the given tag.Ni i i����( R R R3 R� t rangeR~ Rt R� ( R^ R~ R� t inclusivePopt most_recently_poppedt
stack_sizeR� t t( ( s0 /usr/lib/python2.7/site-packages/bs4/__init__.pyt _popToTag� s
c C s� | j � | j rN t | j � d k rN | j j sJ | j j | | � rN d St | | j | | | | | j | j
� } | d k r� | S| j
r� | | j
_ n | | _
| j | � | S( s Push a start tag on to the stack.
If this method returns None, the tag was rejected by the
SoupStrainer. You should proceed as if the tag had not occurred
in the document. For instance, if this was a self-closing tag,
don't call handle_endtag.
i N(
R| R! R3 R� R� t
search_tagR R RJ R} R� R� R� ( R^ R~ R� R� R� R� ( ( s0 /usr/lib/python2.7/site-packages/bs4/__init__.pyt handle_starttag� s
c C s | j � | j | | � d S( N( R| R� ( R^ R~ R� ( ( s0 /usr/lib/python2.7/site-packages/bs4/__init__.pyt
handle_endtag s
c C s | j j | � d S( N( R� R� ( R^ t data( ( s0 /usr/lib/python2.7/site-packages/bs4/__init__.pyt handle_data s t minimalc C sp | j r5 d } | d k r( d | } n d | } n d } | sJ d } n d } | t t | � j | | | � S( sl Returns a string or Unicode representation of this document.
To get Unicode, pass None for encoding.R s encoding="%s"u <?xml version="1.0"%s?>
u i N( R> R t superR Rw ( R^ t pretty_printt eventual_encodingt formattert
encoding_partRt t indent_level( ( s0 /usr/lib/python2.7/site-packages/bs4/__init__.pyRw s
N( t __name__t
__module__t __doc__R R8 R� RI R Rj Rm Rp t staticmethodRV R] R\ R� R
R� R� R� R� R� R| R� R� R� R� R� R� RS R Rw ( ( ( s0 /usr/lib/python2.7/site-packages/bs4/__init__.pyR 8 s6 �
9 t BeautifulStoneSoupc B s e Z d Z d � Z RS( s& Deprecated interface to an XML parser.c O s4 d | d <t j d � t t | � j | | � d S( Nt xmlR_ sx The BeautifulStoneSoup class is deprecated. Instead of using it, pass features="xml" into the BeautifulSoup constructor.( R R R� R� Rj ( R^ t argsR ( ( s0 /usr/lib/python2.7/site-packages/bs4/__init__.pyRj 5 s
( R� R� R� Rj ( ( ( s0 /usr/lib/python2.7/site-packages/bs4/__init__.pyR� 2 s t StopParsingc B s e Z RS( ( R� R� ( ( ( s0 /usr/lib/python2.7/site-packages/bs4/__init__.pyR� = s R: c B s e Z RS( ( R� R� ( ( ( s0 /usr/lib/python2.7/site-packages/bs4/__init__.pyR: @ s t __main__(( R� t
__author__t __version__t
__copyright__t __license__t __all__RO t reR? t tracebackR RJ R R t dammitR t elementR R R R R R
R R R
R R R t _st _soupR� RU R� RA R: R� t stdinRL t prettify( ( ( s0 /usr/lib/python2.7/site-packages/bs4/__init__.pyt <module> s2 L
� �