国产av日韩一区二区三区精品,成人性爱视频在线观看,国产,欧美,日韩,一区,www.成色av久久成人,2222eeee成人天堂

Table of Contents
前言
記住一行代碼
最后
Home Backend Development Python Tutorial A must-have Python crawler tool for lazy people

A must-have Python crawler tool for lazy people

Apr 11, 2023 pm 09:52 PM
python Reptile Artifact

A must-have Python crawler tool for lazy people

前言

今天把最近的一個(gè)應(yīng)用做好了,測(cè)試了一下運(yùn)行沒(méi)有問(wèn)題,剩下的就是檢驗(yàn)一下結(jié)果如何.從光譜到Lab值通常使用matlab中的roo2lab(),不過(guò)經(jīng)過(guò)我最近的測(cè)試發(fā)現(xiàn)轉(zhuǎn)換的結(jié)果并不理想,而且這個(gè)轉(zhuǎn)化的代碼也不是我寫的所以另尋他法,找到了下面這個(gè)網(wǎng)頁(yè)。

A must-have Python crawler tool for lazy people

動(dòng)手

有了這個(gè)網(wǎng)頁(yè),很簡(jiǎn)單就想到去解析.然后很快找到了這個(gè)api,可以看到用post提交表單請(qǐng)求就可以返回結(jié)果。

A must-have Python crawler tool for lazy people


于是一番操作,修改form表單,構(gòu)造傳入的spectrum,但是最終請(qǐng)求得到的卻是一個(gè)頁(yè)面,并不是想要的json.然后許久沒(méi)寫爬蟲的我直接惱火,想到自動(dòng)化工具模擬操作.可是selenium很難用而且還得去找瀏覽器新版本的驅(qū)動(dòng),隨后直接搜索一番,發(fā)現(xiàn)了這個(gè)神器—playwright。

首先老規(guī)矩去它的首頁(yè)看看教程

A must-have Python crawler tool for lazy people

接著安裝一下playwright以及瀏覽器的驅(qū)動(dòng)。

pip install pytest-playwright
playwright install

第二步的時(shí)候很慢,所以我只等他下載好了chrome和ffmpeg直接就ctrl+c停止了,畢竟我也用不著其他的瀏覽器驅(qū)動(dòng)。

然后稍微瀏覽一下這個(gè)文檔功能非常豐富,不過(guò)我用到的功能也不多,接下來(lái)的使用才是真正適合我們懶人的。

記住一行代碼

python -m playwright codegen xxx.com

其中xxx.com就是我們的目標(biāo)網(wǎng)址,運(yùn)行后會(huì)創(chuàng)建一個(gè)熟悉的自動(dòng)化頁(yè)面,然后我們就進(jìn)行一些我們想要的操作,比如設(shè)置開始的最小波長(zhǎng)為400nm,然后觀察以及光源改為D65/10.一系列操作后會(huì)看到對(duì)應(yīng)的代碼已經(jīng)生成好了。

A must-have Python crawler tool for lazy people

至今為止我還沒(méi)有寫一行代碼,不過(guò)構(gòu)造輸入的光譜還是得自己來(lái)的。

把這一段代碼復(fù)制下來(lái),然后自己寫構(gòu)造輸入的函數(shù)(就幾行),然后通過(guò)選擇器(直接左鍵元素復(fù)制xpath)得到lab值,這樣目標(biāo)就搞定了。

整體代碼如下:

import time
from playwright.sync_api import Playwright, sync_playwright, expect
import numpy as np
data_test=np.loadtxt('./dist/1_res.csv',delimiter=',')
def get_str(arr):
arr_str=""
for i in arr:
arr_str+=str(format(i,".2f"))+"rn"
return arr_str
labs=[]
def run(playwright: Playwright) -> None:
browser = playwright.chromium.launch(headless=False)
context = browser.new_context()
# Open new page
page = context.new_page()
# Go to https://www.qtccolor.com/secaiku/tool/spectrum
page.goto("https://www.qtccolor.com/secaiku/tool/spectrum")
# Click div[role="tab"]:has-text("光譜數(shù)據(jù)")
page.locator("div[role="tab"]:has-text("光譜數(shù)據(jù)")").click(
# Click text=最小波長(zhǎng):nm
page.locator("text=最小波長(zhǎng):nm").click()
# Fill [placeholder="33 80"]
page.locator("[placeholder="\33 80"]").fill("400")
# Select 1964
page.locator("select[name="obs"]").select_option("1964")
# Select D65
page.locator("select[name="ill"]").select_option("D65")
# Fill textarea[name="spectrum"]
for i in range(len(data_test)):
inputs=get_str(data_test[i,:])
# Click textarea[name="spectrum"]
page.locator("textarea[name="spectrum"]").click()
page.locator("textarea[name="spectrum"]").press("Control+a")
page.locator("textarea[name="spectrum"]").fill(inputs)
# Click button:has-text("轉(zhuǎn)換顏色")
page.locator("button:has-text("轉(zhuǎn)換顏色")").click()
time.sleep(1)
# Click text=Lab0.000.000.00 >> td >> nth=1
L=float(page.locator('xpath=//*[@]/div[1]/div/div[2]/table/tbody/tr[2]/td[2]').inner_text())
# Click text=Lab0.000.000.00 >> td >> nth=2
a=float(page.locator('xpath=//*[@]/div[1]/div/div[2]/table/tbody/tr[2]/td[3]').inner_text())
# Click text=Lab0.000.000.00 >> td >> nth=3
b=float(page.locator('xpath=//*[@]/div[1]/div/div[2]/table/tbody/tr[2]/td[4]').inner_text())
print(L,a,b)
labs.append([L,a,b])
# ---------------------
context.close()
browser.close()
with sync_playwright() as playwright:
run(playwright)
np.savetxt('./1_lab_res.csv',labs,delimiter=",")

可以說(shuō)從安裝到實(shí)現(xiàn)就幾分鐘,而且特別容易上手,我第一次用也一下就能實(shí)現(xiàn)效果。

A must-have Python crawler tool for lazy people

剩下的就是簡(jiǎn)單的寫個(gè)函數(shù)計(jì)算色差啥的就沒(méi)難度了。

最后

大廠出品果然不同,使用它在不考慮運(yùn)行效率(有異步但是我懶得看了)的情況下可以輕松實(shí)現(xiàn)復(fù)雜操作,懶人最愛!

The above is the detailed content of A must-have Python crawler tool for lazy people. For more information, please follow other related articles on the PHP Chinese website!

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undress AI Tool

Undress AI Tool

Undress images for free

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

Hot Topics

PHP Tutorial
1502
276
PHP calls AI intelligent voice assistant PHP voice interaction system construction PHP calls AI intelligent voice assistant PHP voice interaction system construction Jul 25, 2025 pm 08:45 PM

User voice input is captured and sent to the PHP backend through the MediaRecorder API of the front-end JavaScript; 2. PHP saves the audio as a temporary file and calls STTAPI (such as Google or Baidu voice recognition) to convert it into text; 3. PHP sends the text to an AI service (such as OpenAIGPT) to obtain intelligent reply; 4. PHP then calls TTSAPI (such as Baidu or Google voice synthesis) to convert the reply to a voice file; 5. PHP streams the voice file back to the front-end to play, completing interaction. The entire process is dominated by PHP to ensure seamless connection between all links.

How to use PHP combined with AI to achieve text error correction PHP syntax detection and optimization How to use PHP combined with AI to achieve text error correction PHP syntax detection and optimization Jul 25, 2025 pm 08:57 PM

To realize text error correction and syntax optimization with AI, you need to follow the following steps: 1. Select a suitable AI model or API, such as Baidu, Tencent API or open source NLP library; 2. Call the API through PHP's curl or Guzzle and process the return results; 3. Display error correction information in the application and allow users to choose whether to adopt it; 4. Use php-l and PHP_CodeSniffer for syntax detection and code optimization; 5. Continuously collect feedback and update the model or rules to improve the effect. When choosing AIAPI, focus on evaluating accuracy, response speed, price and support for PHP. Code optimization should follow PSR specifications, use cache reasonably, avoid circular queries, review code regularly, and use X

python seaborn jointplot example python seaborn jointplot example Jul 26, 2025 am 08:11 AM

Use Seaborn's jointplot to quickly visualize the relationship and distribution between two variables; 2. The basic scatter plot is implemented by sns.jointplot(data=tips,x="total_bill",y="tip",kind="scatter"), the center is a scatter plot, and the histogram is displayed on the upper and lower and right sides; 3. Add regression lines and density information to a kind="reg", and combine marginal_kws to set the edge plot style; 4. When the data volume is large, it is recommended to use "hex"

PHP integrated AI emotional computing technology PHP user feedback intelligent analysis PHP integrated AI emotional computing technology PHP user feedback intelligent analysis Jul 25, 2025 pm 06:54 PM

To integrate AI sentiment computing technology into PHP applications, the core is to use cloud services AIAPI (such as Google, AWS, and Azure) for sentiment analysis, send text through HTTP requests and parse returned JSON results, and store emotional data into the database, thereby realizing automated processing and data insights of user feedback. The specific steps include: 1. Select a suitable AI sentiment analysis API, considering accuracy, cost, language support and integration complexity; 2. Use Guzzle or curl to send requests, store sentiment scores, labels, and intensity information; 3. Build a visual dashboard to support priority sorting, trend analysis, product iteration direction and user segmentation; 4. Respond to technical challenges, such as API call restrictions and numbers

python list to string conversion example python list to string conversion example Jul 26, 2025 am 08:00 AM

String lists can be merged with join() method, such as ''.join(words) to get "HelloworldfromPython"; 2. Number lists must be converted to strings with map(str, numbers) or [str(x)forxinnumbers] before joining; 3. Any type list can be directly converted to strings with brackets and quotes, suitable for debugging; 4. Custom formats can be implemented by generator expressions combined with join(), such as '|'.join(f"[{item}]"foriteminitems) output"[a]|[

python connect to sql server pyodbc example python connect to sql server pyodbc example Jul 30, 2025 am 02:53 AM

Install pyodbc: Use the pipinstallpyodbc command to install the library; 2. Connect SQLServer: Use the connection string containing DRIVER, SERVER, DATABASE, UID/PWD or Trusted_Connection through the pyodbc.connect() method, and support SQL authentication or Windows authentication respectively; 3. Check the installed driver: Run pyodbc.drivers() and filter the driver name containing 'SQLServer' to ensure that the correct driver name is used such as 'ODBCDriver17 for SQLServer'; 4. Key parameters of the connection string

python pandas melt example python pandas melt example Jul 27, 2025 am 02:48 AM

pandas.melt() is used to convert wide format data into long format. The answer is to define new column names by specifying id_vars retain the identification column, value_vars select the column to be melted, var_name and value_name, 1.id_vars='Name' means that the Name column remains unchanged, 2.value_vars=['Math','English','Science'] specifies the column to be melted, 3.var_name='Subject' sets the new column name of the original column name, 4.value_name='Score' sets the new column name of the original value, and finally generates three columns including Name, Subject and Score.

Optimizing Python for Memory-Bound Operations Optimizing Python for Memory-Bound Operations Jul 28, 2025 am 03:22 AM

Pythoncanbeoptimizedformemory-boundoperationsbyreducingoverheadthroughgenerators,efficientdatastructures,andmanagingobjectlifetimes.First,usegeneratorsinsteadofliststoprocesslargedatasetsoneitematatime,avoidingloadingeverythingintomemory.Second,choos

See all articles