a >h͗@sddlZddlZddlZddlZddlZddlmZddlmZddl m Z ddl m Z ddl mZddlmZddlmZdd lmZdd lmZmZmZmZdd lmZeeeee d ZGd ddZdS)N)quote)load) InitSystem)JujuSSH)SSHControlPersist)LocalTransport) OCTransport)SaltStackMaster)CommandTimeoutExceptionConnectionExceptionUnsupportedHostExceptionInvalidTransportException)sos_parse_version)localZcontrol_persistocZ saltstackZjujuc@seZdZdhddZeddZddZd d Zd d Zed dZ ddZ ddZ ddZ ddZ diddZeddZddZddZd d!Zd"d#Zd$d%Zd&d'Zd(d)Zd*d+Zd,d-Zdjd.d/Zd0d1Zd2d3Zd4d5Zd6d7Zdkd:d;Zdd?Z d@dAZ!dBdCZ"dDdEZ#dFdGZ$dHdIZ%dJdKZ&dLdMZ'dNdOZ(dPdQZ)dRdSZ*dTdUZ+dVdWZ,dXdYZ-dZd[Z.d\d]Z/d^d_Z0d`daZ1dbdcZ2dddeZ3dfdgZ4dS)lSosNodeNTc Cs||_||_|d|_||d|_|d|_|d|_|d|_|jj |_ |jj |_ d|_ d|_ d|_ d|_i|_i|_|p|jj|_|jjs|jjs|j|j_|r||j_d|_d|_d|_g|_dgggg|dd|_d |_td |_td |_|||_ |j j!r(|j j!d k|_z|j "|jWn8t#yr}z|$d |WYd}~n d}~00|%|_ |j j|_|j r|jj&rd}|j'r|r|j s|j (dS|j r|)rd|j _*|j j*r|+|,dS)NZ cmdlineoptstmpdirhostlen need_sudo sos_optionsFsos_cmd)versionenableddisabledoptionspresetsr sosreportZsosZsos_uirootzUnable to open remote session: )-stripaddresscommonsopts_assign_config_optsrrrrnode_config_fileinherit_config_filerhostclusterhostname sos_env_vars _env_varspassword _passwordZ nopasswd_sudoZsudo_pwsos_path retrievedZhash_retrieved file_listsos_infosos_binloggingZ getLoggersoslogui_log_load_remote_transport _transportZ default_userZconnect Exception log_errordetermine_host_policyZno_local connected disconnectcheck_in_container containerizedcreate_sos_container_load_sos_info)selfrr r*Z local_sudoZ load_factserrrA9/usr/lib/python3.9/site-packages/sos/collector/sosnode.py__init__,st                  zSosNode.__init__cCs|jr|jjSdS)NT)r5r9r?rArArBr9qszSosNode.connectedcCs|jdS)zDWrapper to close the remote session via our transport agent N)r5r:rDrArArBr:xszSosNode.disconnectcCst|jdvrd|_t|j|S|jjtvr|d}|d|j|d<q>|jS)Nz env --nullstatusroutput=rJ)r)rosenvironcopy run_commandsplit)r?retZln_valrArArBenv_varss   zSosNode.env_varscCs`||_|jd|j|jd|jj|jd|jd|jdd|jd|jjdS) z>Set the manifest section that this node will write to r'policyZ sos_versionrfinal_sos_commandrGN)manifest add_fieldrKr%os_release_namer/r5name)r?r^rArArBset_node_manifests zSosNode.set_node_manifestcCs<tjdr|ddStjddur8|ddSdS)zV Tries to identify if we are currently running in a container or not. z/run/.containerenvz/Found /run/.containerenv. Running in container.T containerNz/Found env var 'container'. Running in containerF)rSpathexists log_debugrTgetrDrArArBr;s   zSosNode.check_in_containercCs|jjr|jj|jj||jjd}|j|dd}|ddvr|ddkrd|dvrf|d t d |dvr|d t |j|j dd}|dd kr| d |jj ddS|d|dt |d|dt dS)zJIf the host is containerized, create the container we'll be using )imageZauthZ force_pullT need_rootrO)r}rkzunable to retrieve auth tokenrPzHCould not pull image. Provide either a username and password or authfilezunknown: Not foundz%Specified image not found on registryrzTemporary container z createdz(Could not start container after create: z$Could not create container on host: F) r%r<r=r!rhget_container_authZforce_pull_imagerVr7r6Zrestart_sos_containerlog_infoZsos_container_name)r?cmdresrXrArArBr=sD        zSosNode.create_sos_containercCsB|jjr$|jjd|jj|jjS|jjd|jjp>|jjS)zrDetermine what the auth string should be to pull the image used to deploy our temporary container default) r!Z registry_userr%ZruntimesZfmt_registry_credentialsZregistry_passwordZfmt_registry_authfileZregistry_authfileZcontainer_authfilerDrArArBrls  zSosNode.get_container_authFcCs<z"|jd||d}|ddkWSty6YdS0dS)z3Checks for the presence of fname on the remote nodezstat rirOrFN)rVr6)r?fnamerjrorArArB file_existss  zSosNode.file_existscCs|jrd|jvr|jS|jS)NrE)r'rrDrArArBrKszSosNode._hostnamecCsd}t|d|S)zUAttempts to obfuscate sensitive information in log messages such as passwordsz>(?P(pass|key|secret|PASS|KEY|SECRET).*?=)(?P.*?\s)z\g****** )resub)r?rMZregrArArB_sanitize_log_msgszSosNode._sanitize_log_msgcCs|j||dS)z:Format a ui message that includes host name and formattingN)r3inforNrLrArArBui_msgszSosNode.ui_msgcCs8tdd}d|jd|d|}|j|dS)z#Used to print and log info messagesrJ[:] N)inspectstackrKr2rvr?rMcallerZlmsgrArArBrmszSosNode.log_infocCs8tdd}d|jd|d|}|j|dS)z$Used to print and log error messagesrJrxryrzr{N)r|r}rKr2errorr~rArArBr7 szSosNode.log_errorcCsB||}tdd}d|jd|d|}|j|dS)z$Used to print and log debug messagesrJrxryrzr{N)rur|r}rKr2debug)r?rMrrArArBrfs zSosNode.log_debugcCsR|jdtjddk}|jr(|r(|S|jjr>dt|S|jrNd|S|S)zIf we need to provide a sudo or root password to a command, then here we prefix the command with the correct bits ZUSERrzsu -c zsudo -S ) r)rgrSrTrr! become_rootrr)r?rnZis_rootrArArB _format_cmds  zSosNode._format_cmdc Csd}d}|jjdurN|jj|jj}|durd|d}|dr|d}n8|j|jjddd}|ddkr|d d \}}}|rt |dd kr|d 7}z|d |dd7}Wn4t y}z| d |WYd}~n d}~00||j d<|j dr$| d|j dn$|j|jjks>|dd|_dS|drZd|_|jd}|j|ddd}|ddkr||d |dr|dS)zLQueries the node for information about the installed version of sos N.rreleaseT use_containerrjrOrrP-z.0z#Unable to fully parse sos release: zsos version is z!sos is not installed on this nodeFz4.0z sos reportz -l3.6)r%Zcontainer_version_commandpackage_managerZ pkg_versionZ sos_pkg_namejoinrVrrWlenr6rfr/rmrr!Zprimaryr7r9check_sos_versionr0_load_sos_plugins_load_sos_presets) r?verrelpkgZpkgs_r@rnsosinforArArBr>#sH    &      zSosNode._load_sos_infocCsn|jd}|j|ddd}|ddkrj|dD]4}|dr4|dd}|jd |q4dS) Nz --list-presetsTrrOrrPzname:rJr)r0rV splitlinesr startswithrWr/append)r?rnrolineZpnamerArArBrWs  zSosNode._load_sos_presetsc Csd}d}d}d}d}|d|}|d|}|d|} |dd} ||||jd<||||jd <|| ||jd <|| |d |jd <dS) Nz,The following plugins are currently enabled:z-The following plugins are currently disabled:z4The following options are available for ALL plugins:z+The following plugin options are available:z Profiles:z(.*?)z rrrTZprofiles)_regex_sos_helpr/) r?rZENABLEDZDISABLEDZ ALL_OPTIONSZOPTIONSZPROFILESZ enableregZdisregZoptregZproregrArArBr`s    zSosNode._load_sos_pluginsc Csg}t||tjD]}|D]|}|s~z|}rD||dWqtyz}z|d|WYd}~qd}~00q |d} |dd| Dq q|S)NrzError parsing sos help: ,css|]}|r|VqdSN)r).0prArArB }z*SosNode._regex_sos_help..) rsfindallSrrWrr6rfextend) r?ZregexrZis_listroresultrZlsr@rrArArBrqs  ( zSosNode._regex_sos_helpc Cs`z|d||j|WStyZ}z$|d|d|WYd}~dSd}~00dS)z1Reads the specified file and returns the contentsz Reading file zException while reading : Nr])rmr5 read_filer6r7)r?Zto_readr@rArArBrs zSosNode.read_filec Csz|jr(|d|jdj|jdSti|jjtd|jj | dd}|rh|d|jd|S| dt d S) z[Attempts to identify the host installation against supported distributions zusing local policy r[Tz/etc/os-release)cachesysrootinitZ probe_runtimeZ remote_execZ remote_checkzloaded policy z for hostz4Unable to determine host installation. Ignoring nodeN) rrmr r`rr!rrr5rVrr7r )r?r%rArArBr8s  zSosNode.determine_host_policyc CsXz|jd}t|t|kWStyR}z|d|WYd}~dSd}~00dS)aChecks to see if the sos installation on the node is AT LEAST the given ver. This means that if the installed version is greater than ver, this will still return True :param ver: Version number we are trying to verify is installed :type ver: ``str`` :returns: True if installed version is at least ``ver``, else False :rtype: ``bool`` rzError checking sos version: NF)r/rr6r7)r?rZ _node_verr@rArArBrs  zSosNode.check_sos_versioncCs|js dS|jj|duS)z2Checks if a given package is installed on the nodeFN)r%rZ pkg_by_name)r?rrArArB is_installedszSosNode.is_installedrFc Cs|jsr|jsr|dz$|j|j}|s8|dtWn6typ}z|d|WYd}~n d}~00|r|jj r|j |}|r| |}|r|j } | |}|j|||||S)aRuns a given cmd, either via the SSH session or locally :param cmd: The full command to be run :type cmd: ``str`` :param timeout: Time in seconds to wait for `cmd` to complete :type timeout: ``int`` :param use_shell: If a shell is needed to run `cmd`, set to True :type use_shell: ``bool`` or ``auto`` for transport-determined :param use_container: Run this command in a container *IF* the host is a containerized host :type use_container: ``bool`` :param env: Pass environment variables to set for this `cmd` :type env: ``dict`` z-Node is disconnected, attempting to reconnectzFailed to reconnect to nodez!Error while trying to reconnect: N)r9rrfr5Z reconnectr+r r6r%r<Zformat_container_commandrrZupdaterV) r?rntimeout use_shellrjrenvZ reconnectedr@Z_cmd_envrArArBrVs(       zSosNode.run_commandc Csxz6|}|r||n |d|jr4||_Wn4tyj}z|d|WYd}~n d}~00|dS)z.Run an sos report on the node, then collect itz'Unable to determine path of sos archivezError during sos execution: N)execute_sos_commandfinalize_sos_pathr7r,retrieve_sosreportr-r6cleanup)r?rdr@rArArBrs  &zSosNode.sosreportcCs||jdvS)z/Verifies if the given preset exists on the noderr/)r?presetrArArB_preset_existsszSosNode._preset_existscs&tfdd|jd|jdfDS)z/Verifies if the given plugin exists on the nodec3s|]}|vVqdSrrA)rspluginrArBrrz)SosNode._plugin_exists..rr)anyr/r?rrArrB_plugin_existsszSosNode._plugin_existscCs||jdvS)z6Checks to see if the plugin is default enabled on noderrrrArArB_check_enabledszSosNode._check_enabledcCs||jdvS)z7Checks to see if the plugin is default disabled on noderrrrArArB_check_disabledszSosNode._check_disabledcCsR|dd}||sdS||r6||jjvr6dS||rN||jdvSdS)zAttempts to verify that the given option is available on the node. Note that we only get available options for enabled plugins, so if a plugin has been force-enabled we cannot validate if the plugin option is correct or notrrFrT)rWrrr!enable_pluginsrr/)r?optplugrArArB_plugin_option_existss    zSosNode._plugin_option_existscsdfdd|DS)z^Returns a comma delimited list for sos plugins that are confirmed to exist on the noderc3s|]}|r|VqdSrrrorDrArBrrz,SosNode._fmt_sos_opt_list..)r)r?r!rArDrB_fmt_sos_opt_list szSosNode._fmt_sos_opt_listcCs ||_dS)zVExpose the node to the cluster profile determined for the environment N)r&)r?r&rArArB set_clusterszSosNode.set_clustercs^|jjr0|js|jj|_n|d|jjd|jjr\|jjD]}||jvr@|j|q@|jjr|jjD] |jvrl|jj|j<ql|jjr|jjD]>t fdd|j Dsd|jj}|j |q|j |r$|jj |j |Wdn1s0Yn6|jj |j|Wdn1sP0YdS)a+This is used to modify the sos report command run on the nodes. By default, sos report is run without any options, using this will allow the profile to specify what plugins to run or not and what options to use. This will NOT override user supplied options. zCluster specified preset z> but user has also defined a preset. Using user specification.c3s|]}|vVqdSrrArrrArBr2rz2SosNode.update_cmd_from_cluster..rRN)r&Z sos_presetrrmZ sos_pluginsrrrZsos_plugin_optionsrplugoptsZcheck_node_is_primarylockZset_primary_optionsZset_node_options)r?roptionrArrBupdate_cmd_from_clusters0        . zSosNode.update_cmd_from_clustercCsJt|jj|_t|jj|_t|jj|_t|jj|_t|jj|_dS)zFrom the global opts configuration, assign those values locally to this node so that they may be acted on individually. N)listr! only_plugins skip_pluginsrrrrDrArArBr">s zSosNode._assign_config_optscsjd}}|r*|dt|d}g}dr`jjdkr`|dttjjdrjjr|dttjjdr܈jj r|d jj r|d jj r|d tjj d r8jj r|d td jj jjr8|dtd jjdrjjjrj|dttjjdsdrˆjjdkr|djjjjr|djjdrjjr|djjjjr|d|dtj jjj}jD]0}j|}|d|d|rN|ndq*jrfddjD}|r|dtd d d!|Djrjr|d"tjnd#jd$j rfd%dj D}t!|t!j krsz,SosNode.finalize_sos_cmd..z-k css|] }|VqdSrrArrArArBrrz+SosNode.finalize_sos_cmd..z --preset=zRequested to enable preset z" but preset does not exist on nodecsg|]}|r|qSrArrrDrArBrrzRequested plugins z. were requested to be enabled but do not existz--only-plugins=zFinal sos command set to r\Ncsg|]}|r|qSrA)rrrDrArBrrzRequested to skip plugins z%, but plugins are already not enabledz--skip-plugins=cs.g|]&}|jvr|r|r|qSrA)rrrrrDrArBrs zRequested to enable plugins z1, but plugins are already enabled or do not existz--enable-plugins=)+r/determine_sos_labelrrr!threadsrstrZplugin_timeoutZallow_system_changesZ no_env_varsZsinceZ skip_commandsrZ skip_filesZ cmd_timeoutZcontainer_runtimeZ namespacesZ journal_sizeZ low_priorityrreplacerSrdr%Z sos_bin_pathr0rrrrrfrrrsetrrrmr^r_rr)r?rlabelZsos_optsrrYr!ZplugsZnot_onlyZonlyskipZnot_skipZskiplnZnot_onenablerArDrBfinalize_sos_cmdIs                     $$   zSosNode.finalize_sos_cmdcCsd}||j|7}|jjr<||s,|jjn d|jj7}|sDdS|d||drdd}nd}|jdd d|}|d |S) z?Determine what, if any, label should be added to the sos reportr]rNzLabel for sos report set to rz--labelz--namerrrR)r&Zget_node_labelr!rrfrrrW)r?rZlcmdrArArBrs  zSosNode.determine_sos_labelcCs^|jj}|r||d}|d}|d|||_|dd|_|jd|jdS)z\Use host facts to determine if we need to change the sos path we are retrieving fromr]rzFinal sos path: /Zcollected_archiveN) r%Zsos_path_striprrWrmr,archiver^r_)r?rdZpstriprArArBrs  zSosNode.finalize_sos_pathcCsT|dkr dS|dkr d|vr dS|dkr,dSt|dkrJ|d ddSd |S) Nrz+sos report process received SIGKILL on noderJZsudozsudo attempt failedz4sos report terminated unexpectedly. Check disk spacer zsos exited with code )rrW)r?rcstdoutrArArBdetermine_sos_errors zSosNode.determine_sos_errorc Cs8|dzd}d}d}|jjr2d|jj}nR|jjr|jsvd|jddd}|j|jj |d|}nd|jj }|r|j d ||_ |j |j |jj d d d |j d }|d d kr|dD]J}t|dr|}|dr|dd}q|dr|d}q|r~|jd|t|dkrP|jddn,t|dkrn|jddn|jddn|jddn:||d |d}|d|d d|dt||WSty|dYnHty2}z.|d||d |WYd!}~n d!}~00d!S)"z2Run sos report and capture the resulting file pathzGenerating sos report...Fr]z--config-file=z/tmp/rrz.confrT)rrrjrrrOrrPz*sosreport-*tar*)z sha256 z md5  rJzThe checksum is: rxchecksum Z checksum_typeZmd5@Zsha256unknownzError running sos report. rc = z msg = zTimeout exceededz'Exception during sos report execution: zError running sos report: N)rwr!r#r$rrrWr5Zcopy_file_to_remoteZ config_filerrVrr(rfnmatchrrr^r_rrrfr6r r7rm)r?rdrZconfig_file_argZ remote_configrorr@rArArBrsr       zSosNode.execute_sos_commandc Cs|jd}tj||dd}zH||rT|d|d||j||WS| d|dWdSt y}z$| d|d |WYd }~dSd }~00d S) z7Copies the specified file from the host to our temp dirrrzCopying remote z to local zAttempting to copy remote file %, but it does not exist on filesystemFzFailed to retrieve rN) rrSrdrrWrrrmr5 retrieve_filerfr6)r?rdZdestdirdestr@rArArBr5s  zSosNode.retrieve_filec Csd|}zt|ddkr:|d|dWdS||rz|d|d|}|j|d d }|d d kWS|d |dWdSty}z$|d|d|WYd}~dSd}~00dS)z|Removes the spciefied file from the host. This should only be used after we have retrieved the file already r]rrzRefusing to remove path z0: appears to be incorrect and possibly dangerousFzRemoving file zrm -f TrirOrz!Attempting to remove remote file rzFailed to remove rN)rrWrrfrrrmrVr6)r?rdrnroerArArB remove_fileDs  zSosNode.remove_filec Cs|js|jjr>z||jWnty<|dYdS0|d|j| dz| |j}Wn0ty}z||WYd}~dSd}~00|r| d|j |j ddd S| d dS) z,Collect the sos report archive from the nodezFailed to make archive readableFzRetrieving sos report from zRetrieving sos report...Nz!Successfully collected sos reportrrTzFailed to retrieve sos report)rr!rmake_archive_readabler,r6r7rmrrwrr.rrW)r?rXr@rArArBrZs&      zSosNode.retrieve_sosreportcCsT|jdus|jrdSd|jvr6|d|jddS||j}|sP|ddS)zvRemove the sos report archive from the node, since we have collected it and it would be wasted space otherwiseNrzNode sos report path z/ looks incorrect. Not attempting to remove pathzFailed to remove sos report)r,rrfrr7)r?ZremovedrArArBremove_sos_archiveps  zSosNode.remove_sos_archivecCsL||jr,dD]}||j|rq,q|j}|rH|j|dddS)zrrrrr8rrrVrrrrrrrrrr"rrrrrrrrrrrrrArArArBr*sr E    (   4    ((   < r)rr|r1rSrsZshlexrZ sos.policiesrZsos.policies.init_systemsrZsos.collector.transports.jujurZ(sos.collector.transports.control_persistrZsos.collector.transports.localrZsos.collector.transports.ocrZ"sos.collector.transports.saltstackr Zsos.collector.exceptionsr r r r Z sos.utilitiesrrHrrArArArB s*