Cannot read property 'length' of undefined in v4 version

[MohamedLamineAllal]

The issue happen with v4 version (tested on v4.0.3, v4.0.5 and the latest v4.1.2)

With the same config and code base it compile with no problem in v3.9

Expected behavior:

No such error! No failing at internal level! (Plus it works in v3.9 and not v4)

text should not come as undefined! Or should be handled without problem!

Actual behavior:
In v3.9 the code compile with the same config with no problem! In v4 i get Cannot read property 'length' of undefined error! And internal failing!

Through the error stack! i get where the error happened and it was at the function computeLineStarts!
text is coming undefined! i did some console logging! And i got what file was been treated! And what text node were printing! The file is all normal and correct! A comment at the end was the last thing which after it the code fail!

Then i added another instruction after it! (A variable assignment const someVar = '';) and the error went up! The last treated nodes before failing came more up! Here bellow the file at which my console.logging stopped at!

File on which the resolution failed at:

https://gist.github.com/MohamedLamineAllal/472cf2043a100cb03244de5fb1138034

When i added const someVar = ""; at the end! the last treated node was } from

export interface IWebSocketDriver {
  readonly protocol: string;
  readonly readyState: number;
  readonly url: string;
  binaryType: BinaryType;
  readonly CONNECTING: number;
  readonly OPEN: number;
  readonly CLOSING: number;
  readonly CLOSED: number;
  setGetStreamObj: (getStreamObj: () => WebSocket | IWebSocketDriver) => IWebSocketDriver;
  onInit: () => IWebSocketDriver;
  onopen: ((this: IWebSocketDriver, ev: Event) => any) | null;
  onmessage: ((this: IWebSocketDriver, ev: MessageEvent) => any) | null;
  onclose: ((this: IWebSocketDriver, ev: CloseEvent) => any) | null;
  onerror: ((this: IWebSocketDriver, ev: Event) => any) | null;
  close(code?: number, reason?: string): void;
  send(data: string | ArrayBufferLike | Blob | ArrayBufferView): void;
} // <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<!!!!!!!!!!!!!!! here the last treated element

/**
 * THINGS TO DO:
 * =============
 *
 *  decide and define the response output of all the functions
 *
 *  consider a design that add useful information that are exchange platform dependent
 *
 *
 *
 * STREAMS :
 * Need to precise what form the returned object should have
 *
 * ok dodo
 */
const someVar = ''; // <=== added this

How i debugged!?

in tsc.js

ts.stringToToken = stringToToken;
    function computeLineStarts(text) {
        console.log('text: ') // <<<==== here
        console.log(text)
        var result = new Array();
        var pos = 0;

And at

ts.getNonDecoratorTokenPosOfNode = getNonDecoratorTokenPosOfNode;
    function getSourceTextOfNodeFromSourceFile(sourceFile, node, includeTrivia) {
        console.log('Source file: ') // <<<<<<<<<<<<<<<<<<<<<<<<= here
        console.log(sourceFile.path)
        console.log('================+++>')
        if (includeTrivia === void 0) { includeTrivia = false; }
        return getTextOfNodeFromSourceText(sourceFile.text, node, includeTrivia);
    }

First Compilation execution (original file)

Without adding the last const someVar = '';

https://gist.github.com/MohamedLamineAllal/653fd4f597aa821f416a33e86d4514eb

Second compilation (after adding the last line)

https://gist.github.com/MohamedLamineAllal/022f6f285313aec30b82bb72238bcaeb

Always it fails at computeLineStarts() method! And because text is undefined.

Error stack

TypeError: Cannot read property 'length' of undefined
    at computeLineStarts (/home/coderhero/.nvm/versions/node/v15.0.1/lib/node_modules/typescript/lib/tsc.js:6528:27)
    at Object.getLineStarts (/home/coderhero/.nvm/versions/node/v15.0.1/lib/node_modules/typescript/lib/tsc.js:6581:60)
    at getCurrentLineMap (/home/coderhero/.nvm/versions/node/v15.0.1/lib/node_modules/typescript/lib/tsc.js:81810:59)
    at emitDetachedCommentsAndUpdateCommentsInfo (/home/coderhero/.nvm/versions/node/v15.0.1/lib/node_modules/typescript/lib/tsc.js:85144:94)
    at emitBodyWithDetachedComments (/home/coderhero/.nvm/versions/node/v15.0.1/lib/node_modules/typescript/lib/tsc.js:85007:17)
    at emitSourceFile (/home/coderhero/.nvm/versions/node/v15.0.1/lib/node_modules/typescript/lib/tsc.js:83873:21)
    at pipelineEmitWithHint (/home/coderhero/.nvm/versions/node/v15.0.1/lib/node_modules/typescript/lib/tsc.js:81887:24)
    at noEmitNotification (/home/coderhero/.nvm/versions/node/v15.0.1/lib/node_modules/typescript/lib/tsc.js:80586:9)
    at onEmitNode (/home/coderhero/.nvm/versions/node/v15.0.1/lib/node_modules/typescript/lib/tsc.js:70565:13)
    at onEmitNode (/home/coderhero/.nvm/versions/node/v15.0.1/lib/node_modules/typescript/lib/tsc.js:72333:13)

For the shared links i used typescript Version 4.1.2! Otherwise i tested with v4.0.3 and v4.0.5

tsconfig.json

{
  "compilerOptions": {
    "declaration": true,
    "declarationDir": "./dist",
    "module": "commonjs",
    "noImplicitAny": true,
    "lib": ["ESNext", "DOM"],
    "outDir": "./dist",
    "target": "es6",
    "moduleResolution": "node",
    "resolveJsonModule": true,
    "esModuleInterop": true
  },
  "typedocOptions": {
    "mode": "modules",
    "out": "docs"
  },
  "include": ["src/**/*.ts", "src/**/*.json", "test"],
  "exclude": ["node_modules", "dist", "src/**/*.spec.ts", "src/**/*.test.ts"]
}

Related Issues:

• TypeError: Cannot read property 'length' of undefined in computeLineStarts #40747
• TypeError: Cannot read property 'length' of undefined in watch mode and {allowJS: true} #35074

[christian-bromann]

Pinning typescript back to v4.0.5 has resolved the issue for me.

[highflying]

I've had similar issues with 4.1.2, I've had switch to 4.0.5 (I was trying to upgrade from 3.9.7)

I have allowJs set to true setting it to false stopped the error from happening.

[MohamedLamineAllal]

For me i wasn't setting allowJs at all! I just tested again! I both set allowJs to false! and switched again to 4.0.5! The problem remain the same and so the error!

Going back to 3.9 works

[zhongzhong0505]

I've had same issues with 4.1.2

[zhongzhong0505]

any updates?

[ssddi456]

I've had same issues with 4.1.2 too

[ssddi456]

in my case, after some Investigation, i found the new sourcefile with undefined text property create here:

... ...
    function transformDeclarations(context) {
... ...
        return transformRoot;
... ...
        function transformRoot(node) {
... ...
            else {
                var statements = ts.visitNodes(node.statements, visitDeclarationStatements);
                combinedStatements = ts.setTextRange(factory.createNodeArray(transformAndReplaceLatePaintedStatements(statements)), node.statements);
                refs.forEach(referenceVisitor);
                emittedImports = ts.filter(combinedStatements, ts.isAnyImportSyntax);
                if (ts.isExternalModule(node) && (!resultHasExternalModuleIndicator || (needsScopeFixMarker && !resultHasScopeMarker))) {
                    combinedStatements = ts.setTextRange(factory.createNodeArray(__spreadArrays(combinedStatements, [ts.createEmptyExports(factory)])), combinedStatements);
                }
            }
            /// <<<<< here
            var updated = factory.updateSourceFile(node, combinedStatements, /*isDeclarationFile*/ true, references, getFileReferencesForUsedTypeReferences(), node.hasNoDefaultLib, getLibReferences()); 
            /// <<<<< here
            updated.exportedModulesFromDeclarationEmit = exportedModulesFromDeclarationEmit;
            return updated;
... ...
      }
   }

and if add copy text property in the updateSourceFile function the problem seems fixed

        function cloneSourceFileWithChanges(source, statements, isDeclarationFile, referencedFiles, typeReferences, hasNoDefaultLib, libReferences) {
            var node = baseFactory.createBaseSourceFileNode(297 /* SourceFile */);
            for (var p in source) {
                if (p === "emitNode" || ts.hasProperty(node, p) || !ts.hasProperty(source, p)) {
                    debug && console.log('skip property', p);
                    continue;
                }
                node[p] = source[p];
            }
            node.flags |= source.flags;
            // add this line
            node.text = source.text;
            node.statements = createNodeArray(statements);
            node.endOfFileToken = source.endOfFileToken;
            node.isDeclarationFile = isDeclarationFile;
            node.referencedFiles = referencedFiles;
            node.typeReferenceDirectives = typeReferences;
            node.hasNoDefaultLib = hasNoDefaultLib;
            node.libReferenceDirectives = libReferences;
            node.transformFlags =
                propagateChildrenFlags(node.statements) |
                    propagateChildFlags(node.endOfFileToken);
            return node;
        }

im not sure if this is the proper fix.

[ssddi456]

when i'm trying write a workaround for this problem, i found a strange thing
i add a hook on ts's api

    function patchTransformDeclarations() {

      const oldTransformDeclarations = (ts as any).transformDeclarations;
      const oldCreateSourceFile = (ts as any).createSourceFile;
      (ts as any).transformDeclarations = function (context: ts.TransformationContext) {
        const transformRoot = oldTransformDeclarations(context);
        function wrappedTransformRoot(node: ts.SourceFile) {
          const oldText = node.text;

          const updated = transformRoot(node) as ts.SourceFile;
          if (node.fileName !== (node as any).originSourceFile) {
            console.log(`node.fileName == node.originSourceFile false ${updated.text == undefined}`);
          } else if (updated.text == undefined){
            console.log(`!!!!!!!!!!!!
node.fileName == node.originSourceFile true updated.text == undefined true
!!!!!!!!!!!!`);
          }

          if (updated.text == undefined) {
            // 这种文件的ast似乎不对，根本就是另外一个文件的内容
            // 也许我应该再这里重新transform一下？
            updated.text = fs.readFileSync(updated.fileName, 'utf8');
            const originText = fs.readFileSync((node as any).originSourceFile, 'utf8');
            console.log(
              `node.fileName ${node.fileName}
updated.fileName ${updated.fileName}
node.fileName == updated.fileName ${node.fileName == updated.fileName}
(node as any).originSourceFile ${(node as any).originSourceFile}
oldText == updated.text ${oldText == updated.text}
oldText == originText ${oldText == originText}
`,
              !(oldText == updated.text) && (`oldText: 
${oldText.slice(0, 100)}
------------------------------
updated.text:
${updated.text.slice(0, 100)}
`));
          }
          return updated;
        }
        return wrappedTransformRoot
      };
      (ts as any).createSourceFile = (...args: any[]) => {
        const ret = oldCreateSourceFile(...args);
        ret.originSourceFile = args[0];
        return ret;
      };

      return {
        depatch() {
          (ts as any).transformDeclarations = oldTransformDeclarations;
          (ts as any).createSourceFile = oldCreateSourceFile;
        }
      }
    }

and got this log:

node.fileName == node.originSourceFile false true                                                        
node.fileName d:/projects/kemis-h5/src/renderers/kemis/DatePicker/index.tsx                              
updated.fileName d:/projects/kemis-h5/src/renderers/kemis/DatePicker/index.tsx                           
node.fileName == updated.fileName true                                                                   
(node as any).originSourceFile d:/projects/kemis-h5/src/renderers/kemis/Crud/index.tsx                   
oldText == updated.text false                                                                            
oldText == originText true                                                                               
 oldText:                                                                                                
import './StateCard';                                                                                    
import './index.scss';                                                                                   
import React from 'react';                                                                               
import {                                                                                                 
  RegisterRenderer,                                                                                      
------------------------------                                                                           
updated.text:                                                                                            
import React from 'react';                                                                               
import { DatePicker, List } from 'antd-mobile';                                                          
import { FormItem, FormCo                                                                                
                                                                                                         
node.fileName == node.originSourceFile false true                                                        
node.fileName d:/projects/kemis-h5/src/renderers/business/BusinessText/index.tsx                         
updated.fileName d:/projects/kemis-h5/src/renderers/business/BusinessText/index.tsx                      
node.fileName == updated.fileName true                                                                   
(node as any).originSourceFile d:/projects/kemis-h5/src/renderers/kemis/Text/index.tsx                   
oldText == updated.text false                                                                            
oldText == originText true                                                                               
 oldText:                                                                                                
import React from 'react';                                                                               
import { FormItem, FormControlProps } from '../Form/Item';                                               
import { Input                                                                                           
------------------------------                                                                           
updated.text:                                                                                            
import React from 'react';                                                                               
import { FormItem, FormControlProps } from '../../kemis/Form/Item';                                      
impor                                                                                                    
                                                                                                         
node.fileName == node.originSourceFile false true                                                        
node.fileName d:/projects/kemis-h5/src/renderers/business/BusinessSwitch/index.tsx                       
updated.fileName d:/projects/kemis-h5/src/renderers/business/BusinessSwitch/index.tsx                    
node.fileName == updated.fileName true                                                                   
(node as any).originSourceFile d:/projects/kemis-h5/src/renderers/kemis/Switch/index.tsx                 
oldText == updated.text false                                                                            
oldText == originText true                                                                               
 oldText:                                                                                                
import React from 'react';                                                                               
import { FormItem, FormControlProps } from '../Form/Item';                                               
import { Switc                                                                                           
------------------------------                                                                           
updated.text:                                                                                            
import React from 'react';                                                                               
import { FormItem, FormControlProps } from '../../kemis/Form/Item';                                      
impor                                                                                                    
                                                                                                         
node.fileName == node.originSourceFile false true                                                        
node.fileName d:/projects/kemis-h5/src/renderers/business/BusinessPicker/index.tsx                       
updated.fileName d:/projects/kemis-h5/src/renderers/business/BusinessPicker/index.tsx                    
node.fileName == updated.fileName true                                                                   
(node as any).originSourceFile d:/projects/kemis-h5/src/renderers/kemis/Picker/index.tsx                 
oldText == updated.text false                                                                            
oldText == originText true                                                                               
 oldText:                                                                                                
import React from 'react';                                                                               
import { RegisterRenderer, filter, evalExpression, utils } from '@ke/kemi                                
------------------------------                                                                           
updated.text:                                                                                            
import React from 'react';                                                                               
import { RegisterRenderer, filter, evalExpression, utils } from '@ke/kemi                                

this seems like those SourceFiles with undefined text are in fact with wrong source before tranform to declare file...

[ssddi456]

these sourcefile with modified fileName is from createRedirectSourceFile, which seems work incorrect, it redirect declartion file but redirect other things...
hooks:

(ts as any).createSourceFile = (...args: any[]) => {
        const ret = oldCreateSourceFile(...args);
        ret.originSourceFile = args[0];
        const proxy = new Proxy(ret, {
          set(target, name, value) {
            if (name == 'fileName' && value != target[name]) {
              console.log(`modify source fileName to
${target[name]}
${value}`);
            }
            target[name] = value;
            return true;
          }
        })
        return proxy;
      };

log

modify source fileName to
d:/projects/kemis-h5/node_modules/@types/react-dom/node_modules/@types/react/index.d.ts
d:/projects/kemis-h5/node_modules/@types/react-is/node_modules/@types/react/index.d.ts
modify source fileName to
d:/projects/kemis-h5/src/renderers/kemis/Crud/index.tsx
d:/projects/kemis-h5/src/renderers/kemis/DatePicker/index.tsx
modify source fileName to
d:/projects/kemis-h5/src/renderers/kemis/Text/index.tsx
d:/projects/kemis-h5/src/renderers/business/BusinessText/index.tsx
modify source fileName to
d:/projects/kemis-h5/src/renderers/kemis/Switch/index.tsx
d:/projects/kemis-h5/src/renderers/business/BusinessSwitch/index.tsx
modify source fileName to
d:/projects/kemis-h5/src/renderers/kemis/Picker/index.tsx
d:/projects/kemis-h5/src/renderers/business/BusinessPicker/index.tsx
modify source fileName to
d:/projects/kemis-h5/node_modules/@types/uglify-js/node_modules/source-map/source-map.d.ts
d:/projects/kemis-h5/node_modules/@types/webpack/node_modules/source-map/source-map.d.ts

[ssddi456]

i found the redirect condition is check if has same package id, and the incorrect redirect file is with a package.json at same folder, which have the same package name....

\folderA
    \package.json (with name: "text")
    \index.tsx
\folderB
    \package.json (with name: "text")
    \index.tsx (will be redirect to the index.tsx in folderA)

maybe the redirct action should only happened in node_modules folder?