github/node-DeepResearch
1
0
Fork 0
mirror of https://github.com/jina-ai/node-DeepResearch.git synced 2025-12-26 06:28:56 +08:00
Code Issues Packages Projects Releases Wiki Activity

fix: broken cn chars

Browse Source
This commit is contained in:
Han Xiao 2025-03-30 21:09:37 +08:00
parent 15c6b2a080
commit 9a11d7e686
2 changed files with 9 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
src/agent.ts
View File

@ -41,6 +41,7 @@ import {
import {MAX_QUERIES_PER_STEP, MAX_REFLECT_PER_STEP, MAX_URLS_PER_STEP, Schemas} from "./utils/schemas";
import {formatDateBasedOnType, formatDateRange} from "./utils/date-tools";
import {repairUnknownChars} from "./tools/broken-ch-fixer";
import {fixMarkdown} from "./tools/md-fixer";
async function sleep(ms: number) {
const seconds = Math.ceil(ms / 1000);
@ -958,7 +959,12 @@ But unfortunately, you failed to solve the issue. You need to think out of the b
fixCodeBlockIndentation(
repairMarkdownFootnotesOuter(
await repairUnknownChars(
buildMdFromAnswer(thisStep as AnswerAction), context))
await fixMarkdown(
buildMdFromAnswer(thisStep as AnswerAction),
allKnowledge,
context,
SchemaGen),
context))
),
allURLs)));
} else {

4
src/tools/md-fixer.ts
View File

@ -23,14 +23,14 @@ Your task is to repair the provided markdown content while preserving its origin
6. In the footnote section, keep each footnote items format and repair misaligned and duplicated footnotes. Each footnote item must contain a URL at the end.
7. In the actual content, to cite multiple footnotes in a row use [^1][^2][^3], never [^1,2,3] or [^1-3].
8. Pay attention to the original content's ending (before the footnotes section). If you find a very obvious incomplete/broken/interrupted ending, continue the content with a proper ending.
9. Repair any <EFBFBD><EFBFBD> symbols or other broken characters in the original content by decoding them to the correct content.
9. Repair any <EFBFBD><EFBFBD> symbols or other broken unicode characters in the original content by decoding them to the correct content.
10. Replace any obvious placeholders or Lorem Ipsum values such as "example.com" with the actual content derived from the knowledge.
</rules>
The following knowledge items are provided for your reference. Note that some of them may not be directly related to the content user provided, but may give some subtle hints and insights:
${KnowledgeStr.join('\n\n')}
Directly output the repaired markdown content, preserving HTML tables when exist. No explain, no summary, no analysis. Just the repaired content.
Directly output the repaired markdown content, preserving HTML tables if exist, never use tripple backticks html to wrap html table. No explain, no summary, no analysis. Just the repaired content.
`,
user: mdContent
}