Skip to content

ScreenPlay削除対応 #234

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tegnike merged 2 commits into from
Nov 24, 2024
Merged

ScreenPlay削除対応 #234

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
311c716
ScreenPlay廃止
tegnike Nov 24, 2024
314afb5
感情タグを会話履歴に表示させない
tegnike Nov 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion src/components/chatLog.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import Image from 'next/image'
import { useEffect, useRef } from 'react'
import { EMOTIONS } from '@/features/messages/messages'

import homeStore from '@/features/stores/home'
import settingsStore from '@/features/stores/settings'
Expand Down Expand Up @@ -70,6 +71,9 @@ const Chat = ({
message: string
characterName: string
}) => {
const emotionPattern = new RegExp(`\\[(${EMOTIONS.join('|')})\\]\\s*`, 'g')
const processedMessage = message.replace(emotionPattern, '')

const roleColor =
role !== 'user' ? 'bg-secondary text-white ' : 'bg-base text-primary'
const roleText = role !== 'user' ? 'text-secondary' : 'text-primary'
Expand All @@ -90,7 +94,7 @@ const Chat = ({
</div>
<div className="px-24 py-16 bg-white rounded-b-8">
<div className={`typography-16 font-bold ${roleText}`}>
{message}
{processedMessage}
</div>
</div>
</>
Expand Down
5 changes: 3 additions & 2 deletions src/components/useExternalLinkage.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,20 +4,21 @@ import { useTranslation } from 'react-i18next'
import homeStore from '@/features/stores/home'
import settingsStore from '@/features/stores/settings'
import webSocketStore from '@/features/stores/websocketStore'
import { EmotionType } from '@/features/messages/messages'

///取得したコメントをストックするリストの作成（receivedMessages）
interface TmpMessage {
text: string
role: string
emotion: string
emotion: EmotionType
type: string
}

interface Params {
handleReceiveTextFromWs: (
text: string,
role?: string,
emotion?: string,
emotion?: EmotionType,
type?: string
) => Promise<void>
}
Expand Down
89 changes: 47 additions & 42 deletions src/features/chat/handlers.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { getAIChatResponseStream } from '@/features/chat/aiChatFactory'
import { textsToScreenplay, Message } from '@/features/messages/messages'
import { Message, EmotionType } from '@/features/messages/messages'
import { speakCharacter } from '@/features/messages/speakCharacter'
import { judgeSlide } from '@/features/slide/slideAIHelpers'
import homeStore from '@/features/stores/home'
Expand Down Expand Up @@ -58,12 +58,12 @@ export const speakMessageHandler = async (receivedMessage: string) => {
isCodeBlock = false
}

// 返答内容のタグ部分と返答部分を分離
let tag: string = ''
const tagMatch = remainingMessage.match(/^\[(.*?)\]/)
if (tagMatch?.[0]) {
tag = tagMatch[0]
remainingMessage = remainingMessage.slice(tag.length)
// 返答内容の感情部分と返答部分を分離
let emotion: string = ''
const emotionMatch = remainingMessage.match(/^\[(.*?)\]/)
if (emotionMatch?.[0]) {
emotion = emotionMatch[0]
remainingMessage = remainingMessage.slice(emotion.length)
Comment on lines +61 to +66
Copy link

@coderabbitai coderabbitai bot Nov 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

感情抽出ロジックの改善が必要です

感情の抽出方法について、以下の改善点があります：

  1. 正規表現による抽出は脆弱である可能性があります
  2. デフォルトの感情設定がコード内の複数箇所で繰り返されています

以下のような改善を提案します：

+ const DEFAULT_EMOTION: EmotionType = 'neutral';
+ 
+ const extractEmotion = (text: string): { emotion: EmotionType, remainingText: string } => {
+   const emotionMatch = text.match(/^\[(.*?)\]/);
+   if (!emotionMatch?.[0]) {
+     return { emotion: DEFAULT_EMOTION, remainingText: text };
+   }
+   const emotionText = emotionMatch[1];
+   return {
+     emotion: emotionText as EmotionType,
+     remainingText: text.slice(emotionMatch[0].length)
+   };
+ };

この変更により：

  • 感情抽出ロジックが一元化されます
  • デフォルト値の管理が容易になります
  • 型安全性が向上します

Also applies to: 96-106

}

const sentenceMatch = remainingMessage.match(
Expand Down Expand Up @@ -93,14 +93,17 @@ export const speakMessageHandler = async (receivedMessage: string) => {

// 区切った文字をassistantMessageに追加
assistantMessage.push(sentence)
// タグと返答を結合（音声再生で使用される）
let aiText = tag ? `${tag} ${sentence}` : sentence

const aiTalks = textsToScreenplay([aiText], ss.koeiroParam) // TODO
logText = logText + ' ' + sentence
// em感情と返答を結合（音声再生で使用される）
let aiText = emotion ? `${emotion} ${sentence}` : sentence
logText = logText + ' ' + aiText

speakCharacter(
aiTalks[0],
{
message: sentence,
emotion: emotion.includes('[')
? (emotion.slice(1, -1) as EmotionType)
: 'neutral',
},
() => {
homeStore.setState({
assistantMessage: assistantMessage.join(' '),
Expand Down Expand Up @@ -161,7 +164,7 @@ export const processAIResponse = async (
const reader = stream.getReader()
let receivedMessage = ''
let aiTextLog: Message[] = [] // 会話ログ欄で使用
let tag = ''
let emotion = ''
Copy link

@coderabbitai coderabbitai bot Nov 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

コードの重複を解消する必要があります

processAIResponse関数内の感情処理ロジックがspeakMessageHandlerと重複しています。

先ほど提案したextractEmotion関数を使用して、以下のようにリファクタリングすることを提案します：

- let emotion = ''
+ let emotion: EmotionType = DEFAULT_EMOTION
  
  // 感情抽出部分
- const emotionMatch = receivedMessage.match(/^\[(.*?)\]/)
- if (emotionMatch && emotionMatch[0]) {
-   emotion = emotionMatch[0]
-   receivedMessage = receivedMessage.slice(emotion.length)
- }
+ const { emotion: extractedEmotion, remainingText } = extractEmotion(receivedMessage);
+ emotion = extractedEmotion;
+ receivedMessage = remainingText;

これにより：

  • ロジックの一貫性が保たれます
  • コードの保守性が向上します
  • バグの可能性が減少します

Also applies to: 208-212, 236-249

let isCodeBlock = false
let codeBlockText = ''
const sentences = new Array<string>() // AssistantMessage欄で使用
Expand Down Expand Up @@ -202,11 +205,11 @@ export const processAIResponse = async (
// 先頭の改行を削除
receivedMessage = receivedMessage.trimStart()

// 返答内容のタグ部分と返答部分を分離
const tagMatch = receivedMessage.match(/^\[(.*?)\]/)
if (tagMatch && tagMatch[0]) {
tag = tagMatch[0]
receivedMessage = receivedMessage.slice(tag.length)
// 返答内容の感情部分と返答部分を分離
const emotionMatch = receivedMessage.match(/^\[(.*?)\]/)
if (emotionMatch && emotionMatch[0]) {
emotion = emotionMatch[0]
receivedMessage = receivedMessage.slice(emotion.length)
}

const sentenceMatch = receivedMessage.match(
Expand All @@ -230,18 +233,20 @@ export const processAIResponse = async (
continue
}

// タグと返答を結合（音声再生で使用される）
let aiText = `${tag} ${sentence}`
console.log('aiText', aiText)

const aiTalks = textsToScreenplay([aiText], ss.koeiroParam)
aiTextLog.push({ role: 'assistant', content: sentence })
// 感情と返答を結合（音声再生で使用される）
let aiText = `${emotion} ${sentence}`
aiTextLog.push({ role: 'assistant', content: aiText })

// 文ごとに音声を生成 & 再生、返答を表示
const currentAssistantMessage = sentences.join(' ')

speakCharacter(
aiTalks[0],
{
message: sentence,
emotion: emotion.includes('[')
? (emotion.slice(1, -1) as EmotionType)
: 'neutral',
},
() => {
homeStore.setState({
assistantMessage: currentAssistantMessage,
Expand Down Expand Up @@ -270,15 +275,19 @@ export const processAIResponse = async (
// ストリームが終了し、receivedMessageが空でない場合の処理
if (done && receivedMessage.length > 0) {
// 残りのメッセージを処理
let aiText = `${tag} ${receivedMessage}`
const aiTalks = textsToScreenplay([aiText], ss.koeiroParam)
aiTextLog.push({ role: 'assistant', content: receivedMessage })
let aiText = `${emotion} ${receivedMessage}`
aiTextLog.push({ role: 'assistant', content: aiText })
sentences.push(receivedMessage)

const currentAssistantMessage = sentences.join(' ')

speakCharacter(
aiTalks[0],
{
message: receivedMessage,
emotion: emotion.includes('[')
? (emotion.slice(1, -1) as EmotionType)
: 'neutral',
},
() => {
homeStore.setState({
assistantMessage: currentAssistantMessage,
Expand Down Expand Up @@ -456,7 +465,7 @@ export const handleReceiveTextFromWsFn =
async (
text: string,
role?: string,
emotion: string = 'neutral',
emotion: EmotionType = 'neutral',
type?: string
) => {
if (text === null || role === undefined) return
Expand Down Expand Up @@ -497,11 +506,12 @@ export const handleReceiveTextFromWsFn =
if (role === 'assistant' && text !== '') {
let aiText = `[${emotion}] ${text}`
try {
const aiTalks = textsToScreenplay([aiText], ss.koeiroParam)

// 文ごとに音声を生成 & 再生、返答を表示
speakCharacter(
aiTalks[0],
{
message: text,
emotion: emotion,
},
() => {
homeStore.setState({
chatLog: updateLog,
Expand Down Expand Up @@ -565,14 +575,9 @@ export const handleReceiveTextFromRtFn =
try {
speakCharacter(
{
expression: 'neutral',
talk: {
style: 'talk',
speakerX: 0,
speakerY: 0,
message: '',
buffer: buffer,
},
emotion: 'neutral',
message: '',
buffer: buffer,
},
() => {},
() => {}
Expand Down
78 changes: 3 additions & 75 deletions src/features/messages/messages.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,3 @@
import { VRMExpression, VRMExpressionPresetName } from '@pixiv/three-vrm'
import { KoeiroParam } from '../constants/koeiroParam'

// ChatGPT API
export type Message = {
role: string // "assistant" | "system" | "user";
content?:
Expand All @@ -10,84 +6,16 @@ export type Message = {
audio?: { id: string }
}

const talkStyles = [
'talk',
'happy',
'sad',
'angry',
'fear',
'surprised',
] as const
export type TalkStyle = (typeof talkStyles)[number]
export const EMOTIONS = ['neutral', 'happy', 'angry', 'sad', 'relaxed'] as const
export type EmotionType = (typeof EMOTIONS)[number]

export type Talk = {
style: TalkStyle
speakerX: number
speakerY: number
emotion: EmotionType
message: string
buffer?: ArrayBuffer
}

const emotions = ['neutral', 'happy', 'angry', 'sad', 'relaxed'] as const
type EmotionType = (typeof emotions)[number] & VRMExpressionPresetName

/**
* 発話文と音声の感情と、モデルの感情表現がセットになった物
*/
export type Screenplay = {
expression: EmotionType
talk: Talk
}

export const splitSentence = (text: string): string[] => {
const splitMessages = text.split(/(?<=[。．！？\n])/g)
return splitMessages.filter((msg) => msg !== '')
}

export const textsToScreenplay = (
texts: string[],
koeiroParam: KoeiroParam
): Screenplay[] => {
const screenplays: Screenplay[] = []
let prevExpression = 'neutral'
for (let i = 0; i < texts.length; i++) {
const text = texts[i]

const match = text.match(/\[(.*?)\]/)

const tag = (match && match[1]) || prevExpression

const message = text.replace(/\[(.*?)\]/g, '')

let expression = prevExpression
if (emotions.includes(tag as any)) {
expression = tag
prevExpression = tag
}

screenplays.push({
expression: expression as EmotionType,
talk: {
style: emotionToTalkStyle(expression as EmotionType),
speakerX: koeiroParam.speakerX,
speakerY: koeiroParam.speakerY,
message: message,
},
})
}

return screenplays
}

const emotionToTalkStyle = (emotion: EmotionType): TalkStyle => {
switch (emotion) {
case 'angry':
return 'angry'
case 'happy':
return 'happy'
case 'sad':
return 'sad'
default:
return 'talk'
}
}
Loading
Loading