hjkhjkjhkhjkgjghjhgjhgjghjhghjghjhgjjnbvnvbnvk
bnmbnmbnhjkhjkhjkhjktyutyutyuytutyutyutyhgjgjghjgjghjguytutyry
/
opt
/
alt
/
python37
/
lib
/
python3.7
/
site-packages
/
xray
/
apiclient
/
__pycache__
/
Upload FileeE
HOME
B �'Udif � @ sx d Z ddlZddlZddlZddlZddlZddlZddlm Z ddl m Z m Z ddl mZmZ ddlmZ ddlmZ ddlmZ d d lmZ d d lmZmZm Z d d l m Z d d l m Z d dl!m"Z" d dl#m$Z$m%Z% d dl&m'Z' G dd� d�Z(G dd� d�Z)G dd� de(e)�Z*G dd� de(�Z+G dd� de(e)�Z,G dd � d e(�Z-G d d � d e(�Z.G d d � d e(�Z/G d!d"� d"e(�Z0dS )#zA This module contains class implementing MongoDB API interaction � N)�partial)�List�Any)�Session�Response)� HTTPAdapter)�RequestException)�Retry� )� supported)� api_server�proto�adviser_api_server)� XRayAPIError)�open_local_storage)�Task)�get_xray_exec_user�user_mode_verification)�read_jwt_tokenc @ s� e Zd ZdZdd�eed�dd�Zdd� Zd d � Zeed �d d ��Z ee d �dd��Z ed �dd�Z d)e eed�dd�Zd*e ee d�dd�Zd+ee eee d�dd�Zd,eeed �d d �Zd-ee d �d d!�Zd.eeed"�d#d$�Zd/ee d �d%d&�Zd'd(� ZdS )0�Clientz Base client class � unavailable)�tracing_task_id)� system_idr c C sx || _ || _t�d�| _tdtddg�tdddg�dd�}t|d �}t� | _ | j � t � d �|� t | j j d d �| j _ d S ) N� api_client� �GET�POSTi� i� i� )�total�method_whitelist�status_forcelist�backoff_factor)� max_retriesz://� )�timeout)r �task_id�logging� getLogger�loggerr � frozensetr r �session�mountr r �request)�selfr r Z retry_conf�adapter� r. �J/opt/alt/python37/lib/python3.7/site-packages/xray/apiclient/api_client.py�__init__* s zClient.__init__c C s | j j� d| j� d| j� �S )Nz::z::tracing_task_id=)� __class__�__name__� main_endpointr$ )r, r. r. r/ �__repr__9 s zClient.__repr__c C s | j j� d| j� �S )Nz::)r1 r2 r$ )r, r. r. r/ �__str__< s zClient.__str__)�returnc C s t d��dS )z Base endpoint z$instances are to set their endpoint!N)�NotImplementedError)r, r. r. r/ r3 ? s zClient.main_endpointc C s dt � iS )zL Updated request headers :return: dict with headers zX-Auth)r )r, r. r. r/ �_headersF s zClient._headersc K s$ d| j i}|r|�|� tj�|�S )ze Construct query string :return: string including system_id and given kwargs r )r �update�urllib�parse� urlencode)r, �kwargs�initialr. r. r/ � _query_stringP s zClient._query_stringT)�response�with_status_checkr6 c C s� |j s:| j�d|j� td|j� d|j� �d|jid��n| j�d|j|j|j� |jdkrf|� � }ndd i}|r�|d d kr�td |� ���|S ) z� Perform preprocessing checks of received Response: - is it OK, e.g. 200 OK - was it successful, e.g. status == ok and extract JSON from the Response object :return: full JSON representation of response zServer responded: %sz!Unable to connect to server with �:Z resp_data)�extraz [%s:%s] Response received %s�� �status�okz Received unsuccessful response: ) rF r' �warning�textr � status_code�reason�info�url�json)r, r@ rA �resultr. r. r/ �_preprocess_responseZ s zClient._preprocess_responsec C s^ | j ||d�}y4|d }| j�d|j|� |dkr@td|� ���|S tk rX t� S X dS )z� Check received response :param response: a requests.Response object :return: 'result' dict from JSON representation of response )rA rN z [%s] Received response data %sNzReceived empty result: )rO r' rK rI r �KeyError�dict)r, r@ rA rN �datar. r. r/ �_process_responsew s zClient._process_responseN)�endpoint�payload�log_datarA r6 c C s� | j �d|� |r$|r$| j �d|� y6|dkrB| jj|| jd�}n| jj||| jdd�}W n4 tk r� } z| �||dd�|�W dd}~X Y nX | j||d �S ) a. Perform POST request to given endpoint. Add payload as JSON if given :param endpoint: target URL :param payload: dict with date to POST :param log_data: whether to log POST data or not :return: 'result' dict from JSON representation of response zSending POST request to %szData attached to POST: %sN)�headers�< )rM rW r# z POST failedz'Failed to POST data to X-Ray API server)rA )r' rK r) �postr8 r �_give_xray_exceptionrS )r, rT rU rV rA �resp�er. r. r/ �_post� s z Client._post)rT rV rA c C sh | j �d|� y| jj|| jd�}W n4 tk rX } z| �||dd�|�W d d }~X Y nX | j||d�S )Nz Sending DELETE request to %s)rW z DELETE failedz)Failed to DE:ETE data to X-Ray API server)rA )r' rK r) �deleter8 r rZ rS )r, rT rV rA r[ r\ r. r. r/ �_delete� s zClient._delete)rT r6 c C sz |dkr | j � d| �� � �}| j�d|� y| jj|| jd�}W n4 tk rt } z| �||dd�|�W dd}~X Y nX |S )z� GET request to endpoint or to main endpoint if no endpoint given :param endpoint: target URL :return: a requests Response object N�?zSending GET request to %s)rW z GET failedz(Failed to GET data from X-Ray API server) r3 r? r' rK r) �getr8 r rZ )r, rT r[ r\ r. r. r/ �_raw_get� s zClient._raw_get)rT rA r6 c C s | � |�}| j||d�S )z� GET request to endpoint or to main endpoint if no endpoint given :param endpoint: target URL :return: full dict from JSON representation of response without any processing )rA )rb rO )r, rT rA r[ r. r. r/ � _get_full� s zClient._get_fullc C s | � |�}| �|�S )z� GET request to endpoint or to main endpoint if no endpoint given :param endpoint: target URL :return: 'result' dict from JSON representation of response )rb rS )r, rT r[ r. r. r/ �_get� s z Client._getc C sZ | j jd||d|id� y|jd j}W n ttfk rD |}Y nX t|� d|� d��S )z� Process received exception :param exc: original exception :param api_endpoint: requested endpoint :param log_message: text for logging the error :param exc_message: text for internal exception z %s with %srT )rC r z: z. Please, try again later)r' �error�argsrJ � IndexError�AttributeErrorr )r, �excZ api_endpoint� log_message� exc_message�exc_infor. r. r/ rZ � s zClient._give_xray_exception)T)T)NTT)TT)N)NT)N)r2 � __module__� __qualname__�__doc__�strr0 r4 r5 �propertyr3 rQ r8 r? r �boolr rO rS r] r_ rb rc rd rZ r. r. r. r/ r % s( r c @ s2 e Zd ZdZeed�dd��Zeed�dd�Z dS ) � TaskMixinz1 A mixin class with Task related methods )r6 c C s dS )z( Limit processed fields ) rL rE � client_ip� tracing_by� tracing_count� starttime� ini_location� initial_count� request_count� auto_task�userr. )r, r. r. r/ � task_fields� s zTaskMixin.task_fields)� dict_viewr6 c s, � fdd�|� � D �}|d |d< tf |�S )z@ Turn dictionary structure into valid Task type c s i | ]\}}|� j kr||�qS r. )r} )�.0�k�v)r, r. r/ � <dictcomp>� s z#TaskMixin._task.<locals>.<dictcomp>r r$ )�itemsr )r, r~ � task_viewr. )r, r/ �_task� s zTaskMixin._taskN) r2 rm rn ro rq �tupler} rQ r r� r. r. r. r/ rs � s rs c sN e Zd ZdZeed�dd��Zed�� fdd� Zee d�� fdd� Z � Z S ) � TasksClientz! 'tasks' endpoint client )r6 c C s t � dt� d�S )z& Base endpoint: tasks z://z/api/xray/tasks)r r )r, r. r. r/ r3 s zTasksClient.main_endpointc s t � jdd�S )z� Construct query string. Aimed to get auto tasks only :return: string including system_id and type=auto �auto)�type)�superr? )r, r= )r1 r. r/ r? s zTasksClient._query_stringc s t � �� }� fdd�|D �S )z# Get list of Tasks c s g | ]}� � |��qS r. )r� )r �item)r, r. r/ � <listcomp> s z)TasksClient.get_tasks.<locals>.<listcomp>)r� rd )r, rR )r1 )r, r/ � get_tasks s zTasksClient.get_tasks) r2 rm rn ro rq rp r3 r? r r r� � __classcell__r. r. )r1 r/ r� s r� c s� e Zd ZdZdd�eed�� fdd�Zdd� Zed d �d d ��Ze d d�dd�Z e d �dd�Z eed �dd��Z e d �dd�Ze ed�dd�Zed d�dd �Zed d �d d �Zd d �d d!�Zd d �d"d#�Z� ZS )$� DBMClientzD Client class using local dbm storage instead of remote API r )r )r r c s$ t � j||d� d | _| �� | _d S )N)r r )r� r0 � task_object�_db_open�_db)r, r r )r1 r. r/ r0 " s zDBMClient.__init__c C s | j �� d S )N)r� �close)r, r. r. r/ �__del__( s zDBMClient.__del__z gdbm object)r6 c C s t �dd�S )zC Open dbm DB :return: corresponding object z/root/local_mongo�c)�dbm�openr. r. r. r/ r� + s zDBMClient._db_openN)� post_datar6 c C s t �|�| j| j< dS )z3 Update a DBM task with given data N)rM �dumpsr� r$ )r, r� r. r. r/ r] 3 s zDBMClient._postc C sN yt �| j| j �� �S tt jfk rH } ztd�|�W dd}~X Y nX dS )z: Get saved DBM data :return: dict zFailed to load taskN)rM �loadsr� r$ �decoderP �JSONDecodeErrorr )r, r\ r. r. r/ rd 9 s zDBMClient._getc C s t �� jS )N)�uuid�uuid1�hexr. r. r. r/ �_idC s z DBMClient._idc C s$ | � � }| j|d< tf |�| _| jS )z1 Get saved task :return: r$ )rd r$ r r� )r, Z saved_taskr. r. r/ �get_taskG s zDBMClient.get_task)�taskr6 c C s2 | � � | _| j|_d|_| �|�� � || _| jS )a: Create new task and get unique ID url --> URL client_ip --> IP tracing_by --> time|request_qty tracing_count --> COUNT ini_location --> PATH status --> processing :param task: a Task instance :return: task ID �hold)r� r$ rE r] �as_dictr� )r, r� r. r. r/ �createQ s zDBMClient.create)rw r6 c C s6 | j dkr| �� d| j _|| j _| �| j �� � dS )z� Update started|continued task status --> running starttime --> new timestamp :return: N�running)r� r� rE rw r] r� )r, rw r. r. r/ r9 d s zDBMClient.update)�countr6 c C s6 | j dkr| �� d| j _|| j _| �| j �� � dS )z} Update stopped task status --> stopped tracing_count --> new value :return: N�stopped)r� r� rE rv r] r� )r, r� r. r. r/ �stopq s zDBMClient.stopc C s. | j dkr| �� d| j _| �| j �� � dS )zY Complete tracing task status --> completed :return: N� completed)r� r� rE r] r� )r, r. r. r/ �complete~ s zDBMClient.completec C s | j | j= dS )z6 Delete tracing task :return: N)r� r$ )r, r. r. r/ r^ � s zDBMClient.delete)r2 rm rn ro rp r0 r� � staticmethodr� rQ r] rd r� r r� r� �intr9 r� r� r^ r� r. r. )r1 r/ r� s r� c s e Zd ZdZeed�dd��Zed�� fdd� Zedd�d d �Z edd�d d �Z dd�d d�Z dd�dd�Z e ed�dd��Zeed�dd�Zedd�dd�Zedd�dd�Zdd �eedd �d d �Zedd�d d!�Zdd�d"d#�Zdd�d$d%�Zdd�d&d'�Z� ZS )(� APIClientz% X-Ray task API client class )r6 c C s t � dt� d�S )z% Base endpoint: task z://z/api/xray/task)r r )r, r. r. r/ r3 � s zAPIClient.main_endpointc s$ | j dkrt� j| j d�S t� �� S )z� Construct query string :return: string either including system_id and task_id or system_id only r )r )r$ r� r? )r, r= )r1 r. r/ r? � s zAPIClient._query_stringN)r� r6 c C s< | j � d| �� � �}| �|dd� |�� D ��}|d | _dS )z� POST request to "create a task" API endpoint with given data Obtains a task ID :param post_data: dict with POST data z/create?c S s i | ]\}}|d kr||�qS )rw r. )r r� r� r. r. r/ r� � s z*APIClient._post_create.<locals>.<dictcomp>r N)r3 r? r] r� r$ )r, r� rT Z response_datar. r. r/ � _post_create� s zAPIClient._post_createc C s$ | j � d| �� � �}| �||� dS )z| POST request to "update a task" API endpoint with given data :param post_data: dict with POST data z/update?N)r3 r? r] )r, r� rT r. r. r/ � _post_update� s zAPIClient._post_updatec C s. | j dd� }|� d| �� � �}| �|� dS )z< GET request to "share a task" API endpoint N���z/share-request?)r3 r? rd )r, Zshare_endpointrT r. r. r/ �_share� s zAPIClient._sharec C s" | j � d| �� � �}| �|� dS )z> POST request to "delete a task" API endpoint z/delete?N)r3 r? r] )r, rT r. r. r/ r_ � s zAPIClient._deletec C s | � | �� �S )z1 Get saved task :return: )r� rd )r, r. r. r/ r� � s zAPIClient.get_task)r� r6 c s, d|_ � �� fdd�|�� �� D �� � jS )aH Create new task and get unique ID url --> URL client_ip --> IP tracing_by --> time|request_qty tracing_count --> COUNT ini_location --> PATH status --> processing :param task: a Task instance :return: updated Task instance r� c s i | ]\}}|� j kr||�qS r. )r} )r r� r� )r, r. r/ r� � s z$APIClient.create.<locals>.<dictcomp>)rE r� r� r� r$ )r, r� r. )r, r/ r� � s zAPIClient.create)rw r6 c C s | � d|d�� dS )z� Update started|continued task status --> running starttime --> new timestamp :param starttime: time of starting the Task r� )rE rw N)r� )r, rw r. r. r/ r9 � s zAPIClient.update)r� r6 c C s | � d|i� dS )zw Update tracing_count only. No status updated tracing_count --> new value :return: rv N)r� )r, r� r. r. r/ �update_count_only� s zAPIClient.update_count_only)rv )rz rv r6 c C s* |dkrd|i}n ||d�}| � |� dS )a Update tracing_count only. No status updated request_count --> new value tracing_count --> new value if given :param request_count: number of requests already traced :param tracing_count: number of requests left to trace Nrz )rz rv )r� )r, rz rv rR r. r. r/ �update_counts_only� s z APIClient.update_counts_onlyc C s | � d|d�� dS )z} Update stopped task status --> stopped tracing_count --> new value :return: r� )rE rv N)r� )r, r� r. r. r/ r� s zAPIClient.stopc C s | � ddi� dS )zY Complete tracing task status --> completed :return: rE r� N)r� )r, r. r. r/ r� s zAPIClient.completec C s | � � dS )z5 Share tracing task :return: N)r� )r, r. r. r/ �share s zAPIClient.sharec C s | � � dS )z6 Delete tracing task :return: N)r_ )r, r. r. r/ r^ s zAPIClient.delete)r2 rm rn ro rq rp r3 r? rQ r� r� r� r_ r r r� r� r� r9 r� r� r� r� r� r^ r� r. r. )r1 r/ r� � s$ r� c @ s2 e Zd ZdZeed�dd��Zedd�dd�ZdS ) � SendClientz) X-Ray requests API client class )r6 c C s t � dt� d�S )z) Base endpoint: requests z://z/api/xray/requests)r r )r, r. r. r/ r3 + s zSendClient.main_endpointN)rR r6 c C s( | j � d| �� � �}| j||dd� dS )zS Send given data to ClickHouse :param data: dict with data r` F)rV N)r3 r? r] )r, rR rT r. r. r/ �__call__2 s zSendClient.__call__) r2 rm rn ro rq rp r3 rQ r� r. r. r. r/ r� & s r� c sh e Zd ZdZeed�dd��Zed�� fdd� Zed�dd�Z eed �d d �Z ee ed �d d�Z � Z S )� UIAPIClientz, X-Ray User plugin API client class )r6 c C s t � dt� d�S )z) Base endpoint: requests z://z /api/xray)r r )r, r. r. r/ r3 @ s zUIAPIClient.main_endpointc s dd� |� � D �}t� jf |�S )z� Construct query string :return: string including system_id and given kwargs, filtered by non-empty values c S s i | ]\}}|d k r||�qS )Nr. )r r� r� r. r. r/ r� M s z-UIAPIClient._query_string.<locals>.<dictcomp>)r� r� r? )r, r= Z filter_empty)r1 r. r/ r? G s zUIAPIClient._query_stringc C s� | j t� d�}| j� d|� �}| �|�}xt|d D ]h}|d dkrDq2tj|d �� dd��� }t|��.}|d d kr�|d |j |d < |j |d < W dQ R X q2W |S )zd Get list of tasks and return not processed (full) response from API server )r| z/tasks?rN rE r� r r" )� digest_sizeru �timery rv rz N) r? r r3 rc �hashlib�blake2b�encode� hexdigestr �processed_requests)r, �qsrT r@ r� �fake_id�storager. r. r/ � get_task_listP s zUIAPIClient.get_task_list)r$ r6 c C s& | j |d�}| j� d|� �}| �|�S )zG Get list of requests collected for given tracing task )r z /requests?)r? r3 rc )r, r$ r� rT r. r. r/ �get_request_listf s z UIAPIClient.get_request_list)r$ � request_idr6 c C s( | j ||d�}| j� d|� �}| �|�S )zU Get collected statistics for given request ID of given tracing task )r r� z /request?)r? r3 rc )r, r$ r� r� rT r. r. r/ �get_request_datan s z UIAPIClient.get_request_data)r2 rm rn ro rq rp r3 r? rQ r� r� r� r� r� r. r. )r1 r/ r� ; s r� c s� e Zd ZdZ� fdd�Zeed�dd��Zeed�dd��Z ed�d d �Z e d d �d d�Z d ee e d�dd�Ze d d �dd�Zed�dd�Zed�dd�Zee d�dd�Zee d�dd �Z� ZS ) �SmartAdviceAPIClientz( X-Ray Adviser API client class c s t � jdd� d S )N� not_needed)r )r� r0 )r, )r1 r. r/ r0 } s z SmartAdviceAPIClient.__init__)r6 c C s dt � d�S )z) Base endpoint: requests zhttps://z/api)r )r, r. r. r/ r3 � s z"SmartAdviceAPIClient.main_endpointc C s dS )z3 Limit fields available for update )rE �sourcerJ r. )r, r. r. r/ �fields_allowed� s z#SmartAdviceAPIClient.fields_allowedc K s6 dd� t D �}t� }|r&|�d|f� tjj|dd�S )za Construct query string :return: string including types and given kwargs c S s g | ] }d |f�qS )r� r. )r �_tr. r. r/ r� � s z6SmartAdviceAPIClient._query_string.<locals>.<listcomp>�username�,)�safe)�supported_advice_typesr �appendr: r; r< )r, r= r> � user_contextr. r. r/ r? � s z"SmartAdviceAPIClient._query_stringN)rR r6 c C s" | j � d�}| j||ddd� dS )z] Send given data to Adviser microservice :param data: dict with data z /requests/addF)rV rA N)r3 r] )r, rR rT r. r. r/ r� � s z SmartAdviceAPIClient.__call__)rT rU r6 c C sj | j �d|� y| jj||| jd�}W n4 tk rZ } z| �||dd�|�W dd}~X Y nX | j|dd�S )z� Perform PATCH request to given endpoint. Add payload as JSON. :param endpoint: target URL :param payload: dict with data to PATCH :return: full response zSending PATCH request to %s)rM rW z PATCH failedz/Failed to PATCH data to Smart Advice API serverNF)rA )r' rK r) �patchr8 r rZ rO )r, rT rU r[ r\ r. r. r/ �_patch� s zSmartAdviceAPIClient._patchc C s | j � d�}| j||dd� dS )z9 Send statistics to Adviser microservice z/requests/metadataF)rA N)r3 r] )r, rR rT r. r. r/ � send_stat� s z SmartAdviceAPIClient.send_statc C s" | j � d| �� � �}| j|dd�S )z$ Get list of advice z /advice/list?F)rA )r3 r? rc )r, rT r. r. r/ � advice_list� s z SmartAdviceAPIClient.advice_listc C s | j � d|� �}| j|dd�S )z> Get urls/advices information per user`s site z/advice/site_info/F)rA )r3 rc )r, r� rT r. r. r/ � site_info� s z SmartAdviceAPIClient.site_info)� advice_idr6 c C s | j � d|� d�}| j|dd�S )z= Get details of an advice by given advice_id z/advice/z/detailsF)rA )r3 rc )r, r� rT r. r. r/ �advice_details� s z#SmartAdviceAPIClient.advice_detailsc s2 � fdd�|� � D �}� j� d|� �}� �||�S )z� Partial update of an advice by given advice_id. Fields allowed for update are limited by fields_allowed property c s i | ]\}}|� j kr||�qS r. )r� )r r� r� )r, r. r/ r� � s z6SmartAdviceAPIClient.update_advice.<locals>.<dictcomp>z/advice/)r� r3 r� )r, r� r= rR rT r. )r, r/ � update_advice� s z"SmartAdviceAPIClient.update_advice)N)r2 rm rn ro r0 rq rp r3 r� r� r? rQ r� r r� r� r r� r� r� r� r� r� r. r. )r1 r/ r� x s r� c s� e Zd ZdZ� fdd�Zed�dd�Zeed�dd��Zde e e d �d d �Z eeed �dd�Z eeed �dd�Zeeed �dd�Zed�dd�Z� ZS )�AWPProvisionAPIClientz( X-Ray Adviser API client class c s t � jdd� d S )Nr� )r )r� r0 )r, )r1 r. r/ r0 � s z AWPProvisionAPIClient.__init__)r6 c K s t j�|�S )N)r: r; r< )r, r= r. r. r/ r? � s z#AWPProvisionAPIClient._query_stringc C s dt � d�S )z) Base endpoint: requests zhttps://z/awp)r )r, r. r. r/ r3 � s z#AWPProvisionAPIClient.main_endpointT)r@ rA r6 c C s | j ||d�S )N)rA )rO )r, r@ rA r. r. r/ rS � s z'AWPProvisionAPIClient._process_response)� account_id�domain�websitec C s& | j � d�}| j||||d�ddd�S )zG Gets pullzone if already exists, otherwise creates it z /cdn/pullzone)r� � original_urlr� F)rV rA )r3 r] )r, r� r� r� rT r. r. r/ �get_create_pullzone� s z)AWPProvisionAPIClient.get_create_pullzonec C s2 | j � d�}| j|� d| j|||d�� �ddd�S )zG Gets pullzone if already exists, otherwise creates it z /cdn/pullzoner` )r� r� r� F)rV rA )r3 r_ r? )r, r� r� r� rT r. r. r/ �remove_pullzone� s z%AWPProvisionAPIClient.remove_pullzonec C s& | j � d�}| j||||d�ddd�S )Nz /cdn/purge)r� r� r� F)rV rA )r3 r] )r, r� r� r� rT r. r. r/ �purge_cdn_cache s z%AWPProvisionAPIClient.purge_cdn_cache)r� c C s" | j � d| j|d�� �}| �|�S )Nz /cdn/usage?)r� )r3 r? rd )r, r� rT r. r. r/ � get_usage s z AWPProvisionAPIClient.get_usage)T)r2 rm rn ro r0 rp r? rq r3 r rr rQ rS r� r� r� r� r� r. r. )r1 r/ r� � s r� )1ro r� r� rM r% � urllib.parser: r� � functoolsr �typingr r �requestsr r Zrequests.adaptersr Zrequests.exceptionsr Z$requests.packages.urllib3.util.retryr Zadviser.advice_typesr r� Zinternal.constantsr r r Zinternal.exceptionsr Zinternal.local_countersr Zinternal.typesr Zinternal.user_plugin_utilsr r Zinternal.utilsr r rs r� r� r� r� r� r� r� r. r. r. r/ �<module> s< E t =e