作業指示データを使用した情報検索
R2023b 以降
この例では、情報検索技術を使用し、過去に実行されたアクションと作業指示書の説明に基づいて新しい作業指示書の解決策を検索する方法を説明します。ここでは、過去のインシデントに関する説明テキストと実行されたアクションを活用して、新しい問題に対して考えられる解決策を提案する方法を示します。
データの読み込み
Traffic Signal Work Orders データセット [1] をダウンロードします。
dataFolder = fullfile(tempdir,"traffic_signal_work_orders"); if datasetExists(dataFolder) zipFile = matlab.internal.examples.downloadSupportFile("textanalytics","data/Traffic_Signal_Work_Orders.zip"); unzip(zipFile,"data"); end
CSV ファイルからデータセットを読み込みます。
filename = fullfile(dataFolder,"Traffic_Signal_Work_Orders.csv"); data = readtable(filename, ... VariableNamingRule="preserve", ... TextType="string");
データの最初の数行を表示します。
head(data)
Work Order ID Status Asset Type Asset ID Location ID Created Date Modified Date Submitted Date Closed Date Fiscal Year Work Type Work Needed Work Type Other Work Requested By Job Description Problem Found Action Taken Follow-Up Needed Child Work Order Parent Work Order Is Follow-Up TMC Issue ID Service Request # Damage Report Location Name Latitude Longitude Location
______________ ________ ________________ ________ ______________ ______________________________ ______________________________ ______________________________ ______________________________ ___________ ________________ _______________________________________________________________ _____________________________________________________________________ _____________________________ _____________________________________________________________________________ ________________________________________________________________________________________________________________________________ __________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ ________________ ________________ _________________ ____________ ______________ _________________ _____________ ______________________________________________________________ ________ _________ ______________________________
"WRK17-001685" "Closed" "School Flasher" NaN <missing> "08/19/2017 08:55:00 PM +0000" "09/14/2017 06:27:00 PM +0000" "08/19/2017 09:00:00 PM +0000" "09/14/2017 06:27:00 PM +0000" 2017 "Scheduled Work" "Call-Back (Test Monitors and Cabinets)" <missing> "Austin Transportation Staff" "HAVE AUSTIN ENERGY TIE IN NEW SOURCE DROP OVERHEAD @ CIMA SERENA WB FLASHER" "N/A." "AUSTIN ENERGY TECHNICIANS DISPATCHED TO LOCATION. AE TECHS COULD NOT DO WORK BECAUSE OF LACK OF METER ON POLE/SOURCE. AE TECHS SAID TO CONTACT "WORK MANAGMENT NORTH" 5125057179 FOR FURTHER ACTION. INFORMATION WILL BE RELAYED TO SUPERVISOR. " "False" <missing> <missing> <missing> <missing> <missing> <missing> <missing> NaN NaN <missing>
"WRK17-001865" "Closed" "Signal" 317 "LOC16-001550" "08/24/2017 03:28:00 PM +0000" "09/14/2017 06:42:00 PM +0000" "08/24/2017 03:56:00 PM +0000" "09/14/2017 06:42:00 PM +0000" 2017 "Scheduled Work" "Installation - Other" <missing> "Austin Transportation Staff" <missing> "bad cable for nb in the conduits" "pulled in 20 conductor cable for nb signals and peds . installed a new 332 cabinet , respliced all signals and peds for 2 way project ." "False" <missing> <missing> <missing> <missing> <missing> <missing> "5TH ST / TRINITY ST" NaN NaN "POINT (-97.739677 30.266132)"
"WRK17-001875" "Closed" "Signal" 319 "LOC16-001560" "08/24/2017 03:45:00 PM +0000" "09/14/2017 06:54:00 PM +0000" "08/24/2017 04:03:00 PM +0000" "09/14/2017 06:54:00 PM +0000" 2017 "Scheduled Work" "Installation - Other" <missing> "MMC" "install wb standard and splice in signals and peds" <missing> "install wb mast arm, remove street light pole, splice signal cables and peds" "False" <missing> <missing> <missing> <missing> <missing> <missing> "5TH ST / RED RIVER ST" NaN NaN "POINT (-97.737488 30.265535)"
"WRK17-001890" "Closed" "School Flasher" NaN <missing> "08/24/2017 08:23:00 PM +0000" "08/24/2017 08:31:00 PM +0000" "08/24/2017 08:31:00 PM +0000" "08/28/2017 03:08:00 PM +0000" 2017 "Trouble Call" "OtherDay-Call (Deliver Timing sheets to intersections and PM)" "SOMMERS ELEMENTARY - NOT FLASHING↵↵SR #17-00242843↵#17-00244051↵" "Austin Transportation Staff" "SOMMERS ELEMENTARY - NOT FLASHING↵SR #17-00242843, #17-00244051" "NO PROBLEMS FOUND AT SCHOOL FLASHERS. BOTH PEDESTRIAN FLASHERS NEED SCHEDULE." "BOTH SCHOOL CLOCKS CHECKED FOR TIME, DATE, SCHEDULE, FLASHERS OPERATION AND COMMUNICATION.↵BOTH PEDESTRIAN FLASHER CLOCKS CHECKED FOR TIME, DATE, SCHEDULE, OPERATION, AND COMM.↵TIME, DATE AND SCHEDULE UPDATED IN PEDESTRIAN FLASHER CLOCKS." "False" <missing> <missing> <missing> <missing> <missing> <missing> <missing> NaN NaN <missing>
"WRK17-003185" "Closed" "Signal" 25 "LOC16-000120" "10/09/2017 07:46:00 PM +0000" "01/23/2023 04:47:00 PM +0000" "10/09/2017 07:49:00 PM +0000" "10/10/2017 04:45:00 PM +0000" 2018 "Scheduled Work" "Installation - Camera" <missing> "MMC" "replace the avidia cctv with a pelco repaired unit" <missing> "replaced the avidia cctv with a repaired pelco task # 2423015000" "False" <missing> <missing> <missing> <missing> <missing> <missing> "MARTIN LUTHER KING JR BLVD / CONGRESS AVE (MLK/Capitol Mall)" NaN NaN "POINT (-97.738106 30.280687)"
"WRK17-003430" "Closed" "Signal" 185 "LOC16-000915" "10/18/2017 08:43:00 PM +0000" "10/26/2017 07:30:00 PM +0000" "10/18/2017 08:49:00 PM +0000" "10/26/2017 07:30:00 PM +0000" 2018 "Trouble Call" "Visibility Issue" <missing> "MMC" "Tree limbs blocking WB signal direction." "Tree limbs blocking WB signal direction." "Cut limbs blocking WB signal direction to make visible for ongoing traffic." "True" <missing> <missing> <missing> "TMC17-006530" "17-00311041" <missing> "LAMAR BLVD / PANTHER TRL" NaN NaN "POINT (-97.789284 30.23867)"
"WRK17-001895" "Closed" "Signal" NaN <missing> "08/24/2017 08:32:00 PM +0000" "08/24/2017 08:40:00 PM +0000" "08/24/2017 08:40:00 PM +0000" "08/28/2017 03:06:00 PM +0000" 2017 "Trouble Call" "OtherDay-Call (Deliver Timing sheets to intersections and PM)" "DOSS/MURCHISON COMBO WB NOT FLASHING" "Austin Transportation Staff" "DOSS/MURCHISON COMBO WB NOT FLASHING" "WB FLASHER ON GREYSTONE DOES NOT HAVE COMMUNICATION. CLOCK HAD NO SCHEDULE. EB FLASHER ON N HILLS DR. HAS LIMBS OBSTRUCTION." "DATE, TIME, SCHEDULE, AND FLASHER OPERATION CHECKED FOR ALL CLCOKS. WB CLOCK ON GREYSTONE PROGRAMMED WITH 2017/2018 SCHEDULE. LIMBS REMOVED FROM EB FLASHER ON N HILLS DR." "False" <missing> <missing> <missing> <missing> <missing> <missing> <missing> NaN NaN <missing>
"WRK17-002010" "Closed" "Signal" 779 "LOC16-003835" "08/29/2017 07:58:00 PM +0000" "09/14/2017 07:04:00 PM +0000" "08/30/2017 11:02:00 AM +0000" "09/14/2017 07:04:00 PM +0000" 2017 "Trouble Call" "Detection Failure" <missing> "MMC" "fisheye camera turned" "gridsmart camera out of alignment" "with assistance from the TMC - realigned camera and tightened" "False" <missing> <missing> <missing> <missing> <missing> <missing> "MC KINNEY FALLS PKWY / WILLIAM CANNON DR" NaN NaN "POINT (-97.72583 30.163218)"
データへのフィルターの適用
この例では、データにフィルターを適用し、"School Beacon" アセット タイプと "Trouble Call" 作業タイプに関連付けられた作業指示書のみを含めます。この例で使用されている "Job Description"、"Problem Found"、および "Action Taken" に列を制限します。
idxSB = data.("Asset Type") == "School Beacon"; idxTC = data.("Work Type") == "Trouble Call"; data = data(idxSB & idxTC,:); data = data(:,["Job Description" "Problem Found" "Action Taken"]);
フィルターが適用されたデータの最初の数行を表示します。
head(data)
Job Description Problem Found Action Taken
____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ ________________________________ ______________________________________________________________________________________________________________
" The director of AISD Transportation Kris received an email from the principal at Becker ES regarding there SZF. They said that Becker’s lights on S. 5th and, I think on Annie, are not flashing on Thursdays or Fridays, but are flashing on Saturday and Sunday. " "NONE." "ALL CLOCKS CHECKED. ALL CLOCKS HAVE SOLID LINK. CLOCK ON ANNIE SCHEDULE CHECKED, NO ISSUES FOUND."
"700 blk of montopolis nb needs a new flasher foundation can mount the flasher and sign to the wood pole temporally until the foundation gets built." <missing> <missing>
<missing> "No Comm" "Checked school schedule and all 3 beacons operation."
"not flashing" "none" "had station 20 turn on WBCU's, checked operation. "
<missing> "School flasher turned" "Turn school flasher straight."
"School zone light pole knocked down overnight" "School zone flasher knockdown." "Rebuilt school zone flasher. Use all original except 2 4" poles. checked operation."
"School zone light pole knocked down overnight" "School zone Flasher knockdown" "SEE OTHER CSR"
<missing> "no issue found" "checked both SZ flashers on Harris Glen, both had correct schedules and time, both LEDs operated in flashers"
解析用のデータの準備
アクセスしやすいように列の名前を変更します。
data.Properties.VariableNames = ["JobDescription" "ProblemFound" "ActionTaken"];
いずれかの列でデータが欠損している場合はその行を削除します。
idxEmptyAT = ismissing(data.ActionTaken); idxEmptyJD = ismissing(data.JobDescription); idxEmptyPF = ismissing(data.ProblemFound); data(idxEmptyAT | idxEmptyJD | idxEmptyPF,:) = [];
"JobDescription" 変数と "ProblemFound" 変数を組み合わせて、新しい変数 "Description" を作成します。このステップは、検索にさらに多くのコンテキストを含めるために実行されます。
data.Description = data.JobDescription + ". " + data.ProblemFound;テキスト データの前処理
Description 変数と ActionTaken 変数を前処理して、ストップ ワードを削除し、小文字に変換し、句読点を消去します。
descriptions = preprocessText(data.Description); actions = preprocessText(data.ActionTaken);
データの可視化
Description 変数と ActionTaken 変数のワード クラウドを表示します。
figure tiledlayout(2,1); nexttile wordcloud(descriptions); title("Description"); nexttile wordcloud(actions); title("Action Taken");
説明とアクションの類似度の評価
bm25Similarity 関数を使用して、前処理された説明とアクションとの間の類似度を計算します。
similarities = bm25Similarity(actions,descriptions); positionsFound = []; for i = 1:size(descriptions,1) [~,idxActions] = sort(similarities(:,i),"descend"); found = idxActions' == i; positionsFound = [found; positionsFound]; end
各問題の説明の類似度ランキングで元のアクションが見つかった位置の平均逆順位 (MRR) を計算します。MRR の値が高いほど、検索中に関連性の高い結果が上位にランク付けされていることになります。
meanReciprocalRank(positionsFound)
ans = 0.3045
新しい問題に対するアクションの取得
新しい問題が与えられたときに、最も高い類似度スコアに基づいて、考えられるアクションを提案します。
newProblem = "One flasher is blinking and the other side is not."; newProblem = preprocessText(newProblem); similaritiesNewProblem = bm25Similarity(actions,newProblem); [~,idxActions] = sort(similaritiesNewProblem,"descend"); data.ActionTaken(idxActions(1:5))
ans = 5×1 string
"replaced AC cube flasher, replaced amber LED, checked operation and also checked operation for Berkman side as well"
"Cut several branches to have better visual of school zone sign. Clear view of sign on top of hill coming from SB side."
"Installed RF relay at the cabinet. Delivered two RF remotes to fire station 30. Checked operation. Relocated flasher to the back side of the mast arm for verification. Relocated the optical sensor to the back of mast arm, Rewired the cabinet. RF signal is too weak. Please see follow-up. Signal is intermittence."
"Called supervisor to confirm it was ok to adjust the span. Called in OCP for traffic control and add another set of eyes for vehicle and tool security. Drilled a hole for another bolt 32 inches above the existing span (East side of street). Removed the span from the old preform. Attached the span to a new preform on the elevated bolt. Tied down cable to the span using tape. The lowest point of the lower flasher is now above 18 feet from the ground."
"Updated TOD for EB flasher and corrected scheduling issue for WB flasher, WB flasher didn't have a schedule for 6/1."
前処理関数
preprocessText 関数は、テキストをトークン化し、ストップ ワードを削除し、テキストを小文字に変換し、句読点を消去します。
function documents = preprocessText(textData) % Tokenize the text. documents = tokenizedDocument(textData); % Remove stop words documents = removeStopWords(documents); % Lower case the text documents = lower(documents); % Erase punctuation. documents = erasePunctuation(documents); end
逆順位関数
reciprocalRank 関数は、ランク付けされたリストが与えられたときに、そのリストの逆順位を計算します。
function rr = reciprocalRank(rankedList) % Find the index of the first relevant item (1 in rankedList) firstRelevant = find(rankedList == 1, 1); % Check if a relevant item was found if isempty(firstRelevant) rr = 0; else rr = 1 / firstRelevant; end end
平均逆順位関数
meanReciprocalRank 関数は、ランク付けされたリストが複数与えられたときに、それらのリストの平均逆順位を計算します。
function mrr = meanReciprocalRank(rankedLists) sumRr = 0; numQueries = size(rankedLists, 1); for i = 1:numQueries sumRr = sumRr + reciprocalRank(rankedLists(i, :)); end % Calculate the mean reciprocal rank mrr = sumRr / numQueries; end
参考文献
Traffic Signal Work Orders. City of Austin Open Data. Retrieved April 30, 2023, from https://data.austintexas.gov/Transportation-and-Mobility/Traffic-Signal-Work-Orders/hst3-hxcz
参考
tokenizedDocument | bm25Similarity
